How to Read a

Research Paper

Tamer ElsayedFaculty of Engineering

Computer Science and Engineering Dept.

May 16th 2013

A Typical Researcher … Will likely spend hundreds of hours every year reading

papers

A “good” graduate student/researcher should read (in average) a paper a day!

References S. Keshav, How to Read a Paper, ACM SIGCOMM

Computer Communication Review, 2007 Philip W. L. Fong, Reading a Computer Science Research

Paper, SIGCSE 2009 Amanda Stent, How to Read a Computer Science

Research Paper, Technical Report.

Mihai Pop, How to Read a Scientific Paper, a Presentation at University of Maryland, College Park.

Whyreading research

papers?

Why? You were asked to

For a literature survey of a new field/problem Be up-to-date on current research in the field Allows you to replicate/extend the results Provides you with useful data Gives you “pre-digested” thoughts To decide whether (and where) to publish Teaches you how to write Review for a conference or a class ….

Roadmap

√ Why reading research papers?

Types of (computing) research papers

Three-pass approach for reading a paper

Doing a literature review

How can I remember the papers I have read?

Types of (Computing) Research Papers

From venue, peer-review, and content perspectives …

Venue?Conference Papers Most recent, “hot off the presses”, information

Technical Reports Expands on the information in a conference paper.

Journal Papers (or Articles)

Expand and combine results from several conference papers

Book Chapters Expand a conference paper or journal article

PhD Dissertations

Frequently revised into either journal papers or book chapters

Posters Extended abstracts, short, good for discussions

Workshop Papers

Very hot/new topics, work in-progress, ideas, preliminary results

Peer-Review? Conference papers and journal papers are “peer-reviewed”

have been examined by other computer scientists (3-5) prior to

publication.

“Double-blind” review

Technical reports are typically not peer-reviewed, but are still

excellent sources of detailed information .

Posters and workshop papers are peer-reviewed (less number

of reviewers).

MSc thesis or PhD dissertation?

Content?• Describe/prove a theory• Describe new algorithmsTheoretical

• Describe an implementation of an algorithm, or part or all of a computer system or application

Engineering

• Describe an experiment designed to test some hypothesisEmpirical

• Review current results in a field of researchSurvey

A Three-pass approach for reading a research paper

Each pass accomplishes specific goals and builds upon the previous pass …

Three-Pass Approach

• Gives you a general idea about the paper.

First pass

• Let you grasp the paper’s content, but not its details.

Second pass • Helps you

understand the paper in depth.

Third pass

The First Pass A quick scan to get a bird’s-eye view of the paper. Should take about 5-10 minutes. Decide whether you need to do any more passes.

1. Carefully read the title, abstract, and introduction2. Read the section and sub-section headings, but ignore

everything else3. Read the conclusion4. Glance over the references, mentally ticking off the

ones you’ve already read

1

At the End of this Pass …You should be able to answer the five Cs:

1. Category: What type of paper is this? A measurement paper? An analysis of an existing system? A description of a research prototype?

2. Context: What is the problem (space)? Which other papers is it related to?

3. Correctness: Do the assumptions appear to be valid?4. Contributions: What are the main contributions?5. Clarity: Is the paper well written?

More passes? 1

The Second Pass Should take up to 1 hour. Read with greater care, but ignore

details such as proofs. Identify key points, or make comments in the margins. Look carefully at the figures, diagrams and other

illustrations in the paper. Remember to mark relevant unread references for

further reading this is a good way to learn more about the background of the

paper.

21

After this Pass … Sometimes, you won’t understand it even at the end of

the second pass subject is new to you, with unfamiliar terminology and acronyms. proof or experimental technique that you don’t understand. poorly written: unsubstantiated assertions and numerous forward

references. it could just be that it’s late at night and you’re tired!

You can now choose to: a) set the paper aside, hoping you don’t need to understand the

material to be successful in your career, b) return to the paper later, perhaps after reading background

material or c) persevere and go on to the third pass. 21

The Third Pass To fully understand the paper (e.g., reviewing) Can take about 4-5 hours for beginners and about 1

hour for an experienced reader. Requires great attention to detail. Attempt to virtually re-implement the paper: making

the same assumptions as the authors, re-create the work.

You should identify and challenge every assumption in every statement.

During this pass, you should also jot down ideas for future work.

321

At the End of this Pass … Should be able to reconstruct the entire structure of the

paper from memory. Should be able to identify its strong and weak points. Should be able to pinpoint implicit assumptions,

missing citations to relevant work, and potential issues with experimental or analytical techniques.

321

Doing a Literature Survey

Doing a Literature Survey

What is it?

Requires you to read tens of papers, perhaps in an

unfamiliar field.

What papers should you read?

3-pass approach to help.

First Step Use an academic search engine and some well-chosen

keywords to find 3 to 5 recent papers in the area. (e.g., Google Scholar, MS Academic Search, CiteSeer, or

ACM digital library) Do one pass on each paper to get a sense of the work,

then read their related work sections. You will find a summary of the recent work

and perhaps, if you are lucky, a recent survey paper. If you can find such a survey, you are done.

Read the survey, congratulating yourself on your good luck.

1

Second Step Find shared citations and repeated authors in the

bibliography. These are the key papers and researchers in that area.

Download the key papers and set them aside.

Then go to the websites of the key researchers and see where they’ve published recently. That will help you identify the top conferences in that field

21

Third Step Go to the website for these top conferences and look

through their recent proceedings. A quick scan will usually identify recent high-quality

related work. These papers, along with the ones you set aside earlier,

constitute the first version of your survey. Make two passes through these papers. If they all cite a key paper that you did not find earlier,

obtain and read it, iterating as necessary.

321

How Can I Remember the Papers I Have

Read?

Remembering Read-Papers Strongly recommend that you make an electronic file

for your own bibliography. A BibTeX file is a good idea.

After reading, if you think it’s worth remembering, write an entry for that paper in your bibliography file.

You should note: authors’ names, paper title, venue, date of publication, and page numbers (if possible).

Add a 2-3 sentence description of the paper in which you summarize for yourself the problem addressed by the paper, the solution proposed or results learned, and main contributions.

Other Resources Reading scientific papers (at Purdue)

http://www.lib.purdue.edu/phys/inst/scipaper.html BibTeX

http://www.bibtex.org/ Mendeley

http://www.mendeley.com/ Zotero – firefox extension reference manager

http://www.zotero.org/ Comparison of reference manager software tools

availablehttp://en.wikipedia.org/wiki/Comparison_of_reference_management_software

This talk is one of a series …

How to Do Research Research methodology How to read a research paper How to review a paper How to write a research paper How to write a lit review How to write theses/dissertations How to be a good graduate student How to give a talk How to present a poster

Workshop next Semester …Stay Tuned

AcknowledgementTo my colleagues :

Maram Hasanain Latifa AlMarri Rana Malhas