1. Reviewing, proposing, and reporting research
Chapter 12
Reviewing, proposing, and
reporting research

2. Comparing research designs
The central features of experiments, surveys, qualitative
methods, and comparative historical methods provide distinct
perspectives even when used to study the same social processes.
Comparing subjects randomly assigned to a treatment and to a
comparison group, asking standard questions of the members
of a random sample, observing while participating in a natural
social setting, recording published statistics on national
characteristics, and reading historical documents involve
markedly different decisions about measurement, causality,
and generalizability.

3. Not one of these methods can reasonably be graded as
superior to the others in all respects, and each varies in
its suitability to different research questions and goals.
Choosing among them for a particular investigation
requires consideration of the research problem,
opportunities and resources, prior research, philosophical
commitments, and research goals.

4. Experimental designs are strongest for testing nomothetic
causal hypotheses (law-like explanations that identify a
common influence on a number of cases or events).
But experimental designs have weaknesses. For most
laboratory experiments, people volunteer as subjects, and
volunteers aren’t like other people, so generalizability is
not good.
Ethical and practical constraints limit your treatments.

5. Surveys, because of their probability sampling and
standardized questions, are excellent for generalizable
descriptive studies of large populations.
But surveys, too, have weaknesses. Survey questionnaires
can measure only what respondents are willing to say.
They rely on the truthfulness of respondents and on their
accuracy in reporting.

6. Qualitative methods allow intensive measurement of new
or developing concepts, subjective meanings, and causal
mechanisms.
But such intensive study is time-consuming, so fewer cases
can be examined.
The impossibility of controlling numerous possible
extraneous influences makes qualitative methods a weak
approach to hypothesis testing.

7. Reviewing research
A good literature review is the foundation for a research
proposal, both in identifying gaps in current knowledge and
in considering how to design a research project.
It is also important to review the literature prior to writing
an article about the research findings—the latest findings
on your topic should be checked, and prior research on
new issues should be consulted.
The following are questions you should ask when critiquing
a social research study:

8. Insert exhibit 12.2

9. The goal of the literature review process is to integrate the
results of your separate article reviews and develop an
overall assessment of the implications of prior research.
The integrated literature review should accomplish three
goals (Hart 1998:186–):
Summarize prior research
Critique prior research
Present pertinent conclusions

10. Summarize prior research
Your summary of prior research must focus on the
particular research questions that you will address, but
you may need also to provide some more general background.
Ask yourself three questions about your summary of the
literature (Pyrczak 2005:51–59):
1. Have you been selective? If there have been more
than a few prior investigations of your research question,
you will need to narrow your focus to the most relevant
and highest quality studies. Don’t cite a large number
of prior articles “just because they are there.”

11. Is the research up-to-date? Be sure to include the latest
research, not just the “classic” studies.
Have you used direct quotes sparingly? In order to focus
your literature review, you need to express the key points
from prior research in your own words. Use direct quotes
only when they are essential for making an important point.

12. Critique prior research
Evaluate the strengths and weaknesses of the prior research.
In addition to all the points you develop as you answer
the “article review questions” in Appendix B, you should
also select articles for review that reflect the work of credible
authors in peer-reviewed journals who have been funded
by reputable sources.
Consider the following questions as you decide how much
weight to give each article (Locke, Silverman, & Spirduso
1998:37–44):

13. How was the report reviewed prior to its publication or
release? Articles published in academic journals go through
a very rigorous review process, usually involving careful
criticism and revision. Top “refereed” journals may accept
only 10% of submitted articles, so they can be very selective.
More confidence can be placed in research results that have
been subject to a more rigorous review.
What is the author’s reputation? Reports by an author or
team of authors who have published other work on the research
question should be given somewhat greater credibility at the
outset.

14. Who funded and sponsored the research? Major federal
funding agencies and private foundations fund only research
proposals that have been evaluated carefully and ranked
highly by a panel of experts. They also often monitor closely the
progress of the research. This does not guarantee that every
such project report is good, but it goes a long way toward
ensuring some worthwhile products. On the other hand,
research that is funded by organizations that have a preference
for a particular outcome should be given particularly close
scrutiny.

15. Present pertinent conclusions
Don’t leave the reader guessing about the implications
of the prior research for your own investigation.
Present the conclusions you draw from the research you
have reviewed.
As you do so, follow several simple guidelines
(Pyrczak, 2005:53–56):

16. Distinguish clearly your own opinion of prior research
from conclusions of the authors of the articles you have
reviewed.
2. Make it clear when your own approach is based on
the theoretical framework you are using rather than on the
results of prior research.
3. Acknowledge the potential limitations of any empirical
research project. Don’t emphasize problems in prior research
that you can’t avoid either.

17. Proposing new research
A well-designed proposal can go a long way toward shaping
the final research report and will make it easier to progress at
later research stages (Locke, Spirduso, & Silverman, 2000).
Every research proposal should have at least six sections:
An introductory statement of the research problem, in
which you clarify what it is that you are interested in studying.
2. A literature review, in which you explain how your problem
and plans build on what has already been reported in the
literature on this topic.

18. 3. A methodological plan, detailing just how you will
respond to the particular mix of opportunities and constraints
you face.
4. A budget, presenting a careful listing of the anticipated costs.
5. An ethics statement, identifying human subjects issues in
the research and how you will respond to them in an ethical
fashion.
6. A statement of limitations, reviewing weaknesses of the
proposed research and presenting plans for minimizing their
Consequences.

19. A research proposal also can be strengthened considerably
by presenting a result of a pilot study of the research question.
This might involve administering the proposed questionnaire
to a small sample, conducting a preliminary version of the
proposed experiment with a group of available subjects, or
making observations over a limited period of time in a setting
like that proposed for a qualitative study.
Careful presentation of the methods used in the pilot study
and the problems that were encountered will impress anyone
who reviews the proposal.

20. Reporting research
The goal of research is not just to discover something but
to communicate that discovery to a larger audience: other
social scientists, government officials, your teachers, the
general public—perhaps several of these audiences.
Whatever the study’s particular outcome, if the research
report enables the intended audience to comprehend the results
and learn from them, the research can be judged a success.
If the intended audience is not able to learn about the study’s
results, the research should be judged a failure—no matter
how expensive the research, how sophisticated its design, or
how much of yourself you invested in it.

21. You began writing your research report when you worked
on the research proposal, and you will find that the final
report is much easier to write, and more adequate, if you
write more material for it as you work out issues during
the project.
The last-minute approach does not work for research reports.

22. Writing and organizing
A successful report must be well organized and clearly written.
Getting to such a product is a difficult but not impossible goal.
Consider the following principles formulated by experienced
writers (Booth, Colomb, & Williams, 1995:150–151):
Respect the complexity of the task and don’t expect to
write a polished draft in a linear fashion. Your thinking will
develop as you write, causing you to reorganize and rewrite.

23. 2. Leave enough time for dead ends, restarts, revisions, and
so on, and accept the fact that you will discard much of what
you write.
3. Write as fast as you comfortably can. Don’t worry about
spelling, grammar, and so on until you are polishing things up.
4. Ask anyone you trust for their reactions to what you have written.
5. Write as you go along, so you have notes and report segments drafted even before you focus on writing the report.

24. It is important to outline a report before writing it, but
neither the organization of the report nor the first written
draft should be considered fixed.
As you write, you will get new ideas about how to organize
the report. Try them out. As you review the first draft, you
will see many ways to improve your writing.
Focus particularly on how to shorten and clarify your statements.
You can ease the burden of writing in several ways:

25. 1. Draw on the research proposal and on project notes.
You aren’t starting from scratch; you have all the material
you’ve written during the course of the project.
2. Refine your word processing skills on the computer so
that you can use the most efficient techniques when
reorganizing and editing.
3. Seek criticism from friends, teachers, or other research
consumers before you turn in the final product. They will
alert you to problems in the research or the writing.

26. Your report should be clearly organized into sections,
probably following a standard format that readers will
immediately understand.
Any research report should include:
1. introductory statement of the research problem
2. a literature review
a methodology section
a findings section
a discussion section
a conclusions section
a bibliography

27. Good critical skills are essential in evaluating research
reports, whether your own or those produced by others.
Being aware of the weaknesses, both in others’ studies and
in your own, is a major strength in itself.
You need to be able to weigh the results of any particular
research, and to evaluate a study in terms of its contribution
to understanding the social world—not in terms of whether
it gives a definitive answer for all time, or is perfectly
controlled, or answers all questions.

28. Much research lacks one or more of the three legs of
validity—measurement validity, causal validity, or
generalizability—and contributes more confusion than
understanding about the social world.
Social research methods are only helpful when the
researchers are committed and honest.
Research methods, like all knowledge, can be used poorly
or well, for good purposes or bad, when appropriate or not.