DILLMAN’S TOTAL DESIGN METHOD Mail and Telephone Surveys TDM is described in detail in Dillman’s book Total Design Method published in 1978. This presentation will be a summary of that book. Gary Masters

Mail and phone surveys were viewed with skepticism in the past. Low return rate of mail surveys Phone access limited Face-to-face was accepted standard Literary Digest debacle 1936 (Landon/Roosevelt) Neither mail nor telephone has been considered anything more than a poor substitute for the much heralded face-to face interview. It seemed that only the most educated would respond to the surveys & return rates were very low The problem with the telephone survey was only the wealthy had access to a phone. Two face-to-face interviews for every one mail survey were completed over one 8 year period Twenty face-to-face for every one phone interview over one 8 year test period. 35% of the population had access to phones in 1936

What is the Total Design Method STEP ONE: Identify each aspect of survey process affecting either quality or quantity of response and modify to obtain best possible responses STEP TWO: Organize survey efforts so design intentions are carried out in complete detail

Total Design Method (Cont.) The first step is guided by a theoretical view about why people respond to questionnaires. The second step is guided by an administrative plan which insures the survey is implemented according to design intentions.

Why People Respond Social Exchange The actions of individuals are motivated by the return these actions are expected to bring from others. People attempt to keep their costs below the rewards they expect to receive. Social Exchange Homans, Blau, Thibaut and Kelley Different from “Economic Exchange” which is an exact measure of the worth of one’s actions. The nature of the return under social cannot be bargained about but must be left to the discretion of the one who owes it. It is assumed that people engage in any activity because of the rewards they hope to reap. All activities they perform incur certain costs.

3 things that must be done to maximize survey response Minimize the costs for responding Maximize the rewards for doing so Establish trust that those rewards will be delivered

Respondent Cost Time Physical or mental effort Revealing personal information Becoming subordinate to the researcher Direct monetary costs

Respondent Rewards Being regarded positively by another person Expression of verbal appreciation Being consulted Making the survey interesting Physical gifts/money Opportunity to make a difference

Respondent Trust One must trust that the other will discharge his/her obligation to return the favor with another. One reason token financial incentives have been effective is the fact that they are a symbol of trust. If the study is tied to a trusted company or university credence is given to the study.

Administrative detail is crucial to obtaining high-quality data Planning Timing Supervision Control

Response Rates Mail survey response rates averaging 74% can be counted on using TDM Phone survey response rate as high as 91% are the norm using TDM

Writing Questions Kind of information sought Question structure Choice of words

Types of information sought Attitudes Beliefs Behavior Attributes

Types of information sought Attitudes Overall the content in this class is: Beliefs Compared to other courses this class is more intellectually challenging–agree-disagree Behavior In a typical week, how many hours do you spend outside of class preparing for this class? Attributes What year in school are you? A = Freshman B = Sophomore C = Junior D = Senior E= Other Meaningful A B C D E Meaningless

Question Structure Open-ended Close-ended with ordered choices Close-ended with unordered choices Partially close-ended

Wording Problems Exhausted Tired Candid Honest Priority Most Important Leisure Free Time Supervision Care

Constructing Mail Questionnaires The mail questionnaire, more than any other type of questionnaire, requires careful construction, for it alone comes under the respondent’s complete control.

Questionnaire Attributes Size Structure Color Paper Cover design Question order

Design

Pretest Questions measuring what is intended? Are all words understood? Questions interpreted similarly by all? Close-ended questions apply to all? Questionnaire creates positive impression? Are questions answered as intended?

Causes of Non-response Wrong address and postage rate Resembled “junk” mail Incomplete instructions Intercepted by another person Temporarily laid it aside Return address misplaced Appeared difficult

Cover Letter The cover letter is virtually the only opportunity the researcher has for anticipating and countering respondent questions.

Cover Letter (what to include) Explain what the study is about Convince respondent study is useful His/her response is important to study Promise confidentiality Reemphasize social usefulness Pressed blue ball point signature

Follow-up Mailings Without follow-up mailings, response rates would be less than half those normally attained by the TDM.

Three Carefully Timed Follow-up Mailings One Week: A postcard reminder sent to everyone. Three Weeks: A letter and replacement questionnaire sent only to non-respondents. Seven Weeks: Final mailing similar to preceding except sent by certified mail

Telephone Questionnaires Design of the telephone questionnaire is shaped by the needs of three audiences: Respondents Interviewers Coders

Questions That Are Too Long Key Word: Building in redundancy to stress the main point of the question

The First Question Close-ended Relevant Interesting Easy to answer Applicable to everyone in the study

Implementing Telephone Surveys The speed with which phone surveys can be executed remains one of its primary virtues.

Two Methods for Selecting Sampling Units Sampling of published directories Calling lists of randomly generated telephone numbers.

An Advance Letter It appears that a prior letter influences the quality of data obtained in household phone surveys.

Which Method is Best? It all depends – and that’s final!