Presentation on theme: "How to Read a Research Paper Tamer Elsayed Faculty of Engineering Computer Science and Engineering Dept. May 16 th 2013."— Presentation transcript:
How to Read a Research Paper Tamer Elsayed Faculty of Engineering Computer Science and Engineering Dept. May 16 th 2013
A Typical Researcher … Will likely spend hundreds of hours every year reading papers A “good” graduate student /researcher should read (in average) a paper a day!
References S. Keshav, How to Read a Paper, ACM SIGCOMM Computer Communication Review, 2007 Philip W. L. Fong, Reading a Computer Science Research Paper, SIGCSE 2009 Amanda Stent, How to Read a Computer Science Research Paper, Technical Report. Mihai Pop, How to Read a Scientific Paper, a Presentation at University of Maryland, College Park.
Why reading research papers?
Why? You were asked to For a literature survey of a new field/problem Be up-to-date on current research in the field Allows you to replicate/extend the results Provides you with useful data Gives you “pre-digested” thoughts To decide whether (and where) to publish Teaches you how to write Review for a conference or a class ….
Roadmap √ √ Why reading research papers? Types of (computing) research papers Three-pass approach for reading a paper Doing a literature review How can I remember the papers I have read?
Types of (Computing) Research Papers From venue, peer-review, and content perspectives …
Venue? Conference Papers Most recent, “hot off the presses”, information Technical Reports Expands on the information in a conference paper. Journal Papers (or Articles) Expand and combine results from several conference papers Book Chapters Expand a conference paper or journal article PhD Dissertations Frequently revised into either journal papers or book chapters Posters Extended abstracts, short, good for discussions Workshop Papers Very hot/new topics, work in-progress, ideas, preliminary results
Peer-Review? Conference papers and journal papers are “peer-reviewed” have been examined by other computer scientists (3-5) prior to publication. “Double-blind” review Technical reports are typically not peer-reviewed, but are still excellent sources of detailed information. Posters and workshop papers are peer-reviewed (less number of reviewers). MSc thesis or PhD dissertation?
Content? Describe/prove a theory Describe new algorithms Theoretical Describe an implementation of an algorithm, or part or all of a computer system or application Engineering Describe an experiment designed to test some hypothesis Empirical Review current results in a field of research Survey
A Three-pass approach for reading a research paper Each pass accomplishes specific goals and builds upon the previous pass …
The First Pass A quick scan to get a bird’s-eye view of the paper. Should take about 5-10 minutes. Decide whether you need to do any more passes. 1. Carefully read the title, abstract, and introduction 2. Read the section and sub-section headings, but ignore everything else 3. Read the conclusion 4. Glance over the references, mentally ticking off the ones you’ve already read 1
At the End of this Pass … You should be able to answer the five Cs: 1. Category: What type of paper is this? A measurement paper? An analysis of an existing system? A description of a research prototype? 2. Context: What is the problem (space)? Which other papers is it related to? 3. Correctness: Do the assumptions appear to be valid? 4. Contributions: What are the main contributions? 5. Clarity: Is the paper well written? More passes? 1
The Second Pass Should take up to 1 hour. Read with greater care, but ignore details such as proofs. Identify key points, or make comments in the margins. Look carefully at the figures, diagrams and other illustrations in the paper. Remember to mark relevant unread references for further reading this is a good way to learn more about the background of the paper. 21
After this Pass … Sometimes, you won’t understand it even at the end of the second pass subject is new to you, with unfamiliar terminology and acronyms. proof or experimental technique that you don’t understand. poorly written: unsubstantiated assertions and numerous forward references. it could just be that it’s late at night and you’re tired! You can now choose to: a) set the paper aside, hoping you don’t need to understand the material to be successful in your career, b) return to the paper later, perhaps after reading background material or c) persevere and go on to the third pass. 21
The Third Pass To fully understand the paper (e.g., reviewing) Can take about 4-5 hours for beginners and about 1 hour for an experienced reader. Requires great attention to detail. Attempt to virtually re-implement the paper: making the same assumptions as the authors, re-create the work. You should identify and challenge every assumption in every statement. During this pass, you should also jot down ideas for future work. 321
At the End of this Pass … Should be able to reconstruct the entire structure of the paper from memory. Should be able to identify its strong and weak points. Should be able to pinpoint implicit assumptions, missing citations to relevant work, and potential issues with experimental or analytical techniques. 321
Doing a Literature Survey
What is it? Requires you to read tens of papers, perhaps in an unfamiliar field. What papers should you read? 3-pass approach to help.
First Step Use an academic search engine and some well-chosen keywords to find 3 to 5 recent papers in the area. (e.g., Google Scholar, MS Academic Search, CiteSeer, or ACM digital library) Do one pass on each paper to get a sense of the work, then read their related work sections. You will find a summary of the recent work and perhaps, if you are lucky, a recent survey paper. If you can find such a survey, you are done. Read the survey, congratulating yourself on your good luck. 1
Second Step Find shared citations and repeated authors in the bibliography. These are the key papers and researchers in that area. Download the key papers and set them aside. Then go to the websites of the key researchers and see where they’ve published recently. That will help you identify the top conferences in that field 21
Third Step Go to the website for these top conferences and look through their recent proceedings. A quick scan will usually identify recent high-quality related work. These papers, along with the ones you set aside earlier, constitute the first version of your survey. Make two passes through these papers. If they all cite a key paper that you did not find earlier, obtain and read it, iterating as necessary. 321
How Can I Remember the Papers I Have Read?
Remembering Read-Papers Strongly recommend that you make an electronic file for your own bibliography. A BibTeX file is a good idea. After reading, if you think it’s worth remembering, write an entry for that paper in your bibliography file. You should note: authors’ names, paper title, venue, date of publication, and page numbers (if possible). Add a 2-3 sentence description of the paper in which you summarize for yourself the problem addressed by the paper, the solution proposed or results learned, and main contributions.
Other Resources Reading scientific papers (at Purdue) BibTeX Mendeley Zotero – firefox extension reference manager Comparison of reference manager software tools available management_software management_software
This talk is one of a series …
How to Do Research Research methodology How to read a research paper How to review a paper How to write a research paper How to write a lit review How to write theses/dissertations How to be a good graduate student How to give a talk How to present a poster Workshop next Semester … Stay Tuned Workshop next Semester … Stay Tuned
Acknowledgement To my colleagues : Maram Hasanain Latifa AlMarri Rana Malhas