1. Research Methodology

2. Experimental Research
Part - 1
Experimental Research
 Experimental Research - An attempt by the researcher to maintain control over all factors that may affect the result of an experiment. In doing this, the researcher attempts to determine or predict what may occur.

3. Experimental Design - A blueprint of the procedure that enables the researcher to test his hypothesis by reaching valid conclusions about relationships between independent and dependent variables. It refers to the conceptual framework within which the experiment is conducted

4. Steps involved in conducting
an experimental study
Identify and define the problem.
Formulate hypotheses and deduce their consequences.
Construct an experimental design that represents all the elements, conditions, and relations of the consequences.
Conduct the experiment.
Compile raw data and reduce to usable form.
Apply an appropriate test of significance.

5. Steps in Creating Experimental Design
Select sample of subjects.
Group or pair subjects.
Identify and control non experimental factors.
Select or construct, and validate instruments to measure outcomes.
Conduct pilot study.
Determine place, time, and duration of the experiment.

6. Essentials of Experimental Research
Manipulation of an independent variable.
An attempt is made to hold all other variables except the dependent variable constant - control.
Effect is observed of the manipulation of the independent variable on the dependent variable - observation.

7. Methods of Experimental Control
Physical Control
Gives all subjects equal exposure to the independent variable.
Controls non experimental variables that affect the dependent variable.
Selective Control
Manipulate indirectly by selecting in or out variables that cannot be controlled.
Statistical Control
- Variables not conducive to physical or selective manipulation may be controlled by statistical techniques.

8. Part - 2 Rating Scales What is Scaling?
Scaling is assigning numbers to indicants of the properties of objects

9. Types of Response Scales
Rating Scales
Ranking Scales
Categorization

10. Types of Rating Scales Simple category Multiple choice, Numerical
single response
Multiple choice
multiple response
Likert scale
Semantic differential
Numerical
Multiple rating
Fixed sum
Stapel
Graphic rating

11. Category Scale
The category scale uses multiple items to elicit a single response.
This uses the nominal scale.
Example: where in northern California do you reside? North Bay, South Bay, East Bay, Peninsula, and other.

12. Likert Scale
This is an interval scale and the differences in the responses between any two points on the scale remain the same.
The responses over a number of items tapping a particular concept or variable are then summated for every respondent.
Example: it is designed to examine how strongly subjects agree or disagree with statements on a 5-point scale for the work.

13. Semantic Differential Scale
It is used to assess respondent’s attitudes toward a particular brand, advertisement, object, or individual.
This is treated as an interval scale.
Several bipolar attributes are identified at the extremes of the scale such as Good-Bad, Strong-Weak, and Hot-Cold.
Example: responsive-unresponsive, beautiful-ugly, and courageous-timid.

14. Numerical Scale It is similar to the semantic differential scale.
The 5-point or 7-point scale are provided.
Example: 7-point, extremely pleased-extremely displeased.

15. Itemized Rating Scale
A 5-point or 7-point scale with anchors, as needed, is provided for each item.
When a neutral point is provided, it is a balanced rating scale, and when it is not, it is an unbalanced rating scale.
Research indicates that a 5-point scale is just as good as any, and that an increase from 5 to 7 or 9 points on a rating scale does not improve the reliability of the ratings.

16. Itemized Rating Scale (cont.)
Example: 5 points, balanced rating scale with a neutral point.
Example: 4 points, unbalanced rating scale without a neutral point.

17. Fixed or Constant Sum Scale
The respondents are here asked to distribute a given number of points across various items.
This is in the nature of an ordinal scale.
Example: In choosing a toilet soap such as fragrance, color, shape, size, and texture of lather.

18. Stapel Scale
This scale simultaneously measures both the direction and intensity of the attitude toward the items under study.
This is does not have an absolute zero point, this is an interval scale.
Example: supervisor’s abilities such as adopting modern technology, product innovation, and interpersonal skill, from +3 to –3.

19. Graphic Rating Scale
A graphical representation helps the respondents to indicate on this scale their answers to a particular question by placing a mark at the appropriate point on the line.
This is an ordinal scale, but might appear to make it look like an interval scale.
It is applied as faces scale such as feeling about their jobs.
Example: rate your supervisor, excellent (10) – all right (5) – very bad (1).

20. Part - 3
Survey Method
Survey is process of collection of data by the researcher through oral, written or electronic medium by asking questions to the people concerned to the problem in some way.

21. Purposes of Surveys Opinions Attitudes – Beliefs – Goals – Efficacies
– Evaluations
– Judgments
– Satisfaction
Attitudes
– Goals
– Endorsements
– Preferences
– Styles
Behaviors
– Consumer
– Education
– Health/Safety
– Voting

22. Types of Surveys Written
– Provides respondents opportunities to reflect
– Allows participants to compose extended responses
Oral
– Permits coverage of many topic areas quickly
– Uses simple answers (i.e., yes/no)
Electronic
– Allows efficient administration
– Permits effective data management
– Provides access to large samples
– Permits administration of large surveys with multiple sections

23. Key Questions To Consider When Asking Questions
Is this item understandable to the participant?
Is this item something the participant can answer?
Is this item something the participant will be willing to answer honestly?
How important is this item compared to others I have to avoid respondent burden?

24. Areas to Consider in Designing Your Survey
What type of survey methods are out there and what can they do for me?
What are the things to consider in maximizing the quality of data with respect to questions I ask?
How do I consider using existing scales and putting it all together?

25. Modes of Survey Delivery

26. Survey Method Options IAQ SAQ CASI/CAPI (and A-CASI) CATI T-ACASI
Including Diary Methods, Mail Surveys
CASI/CAPI (and A-CASI)
CATI
T-ACASI
Internet
(IAQ) Interviewer-Administered Questionnaire
(SAQ) Self-Administered Questionnaire
(CASI) Computer-Assisted Self-Interview/(CAPI) Computer-Assisted Personal Interview (A-CASI: Audio-Computer-Assisted Self-Interview)
(CATI) Computer-Assisted Telephone Interview
(T-ACASI) Telephone Audio-Computer-Assisted Self-Interview

27. Interviewer-Administered Questionnaire (IAQ)
Most flexible and responsive to respondent re: comprehension of question, selection of response options.
Able to handle skip patterns that could confuse the respondent in a self-administered questionnaire.
Human element, rapport & responsiveness also can introduce problems in lack of standardization of delivery, self-presentation.

28. Self-Administered Questionnaire (SAQ)
Allows for greater privacy than face-to-face interviewing
Tends to be associated with higher rates of reporting of sensitive behaviors
Literacy requirements for respondent
Cannot be adjusted to be responsive to needs of individual participant
Cannot handle complex “skip patterns”
Low-cost survey delivery option

29. Diary Methods
Provides optimal potential for recording of accurate rates of target behaviors
Recall
Exploration of variations over time
Allows for question design to examine co-occurrence of key behaviors (e.g., sexual behavior and drug use)
Burden of consistent completion of information means that data may be missing in possibly non-random pattern, or completed at a later time than designed.
NOTE: Can be useful to design for Web-based implementation.

30. Mail Surveys
Many of considerations of SAQ are applicable -- with format even more crucial.
Many respondents report liking the flexibility of completing surveys at their own pace.
Need for clear and easy format to maintain respondent interest and ease of completion.
Critical to have contact information and clear procedures to follow up on respondents to ensure adequate response rates.

31. Computer-Assisted Self-Interview /Personal Interview (CASI/CAPI)
Survey is delivered via computer (can be laptops for maximal flexibility), with respondent keying in responses.
Usual mode is “Audio-CASI” or A-CASI, which involves respondent listening to audio recording of survey as they see items on-screen.
For special PCs with touch screens, can avoid keyboard and use fingers to select choice by touching on-screen.
In CAPI, computer-assisted interviewing methods are used to enhance face-to-face data collection with an interviewer present. The interviewer can often use a smaller computer than a regular laptop for this procedure, including tablet-types, as layout and ease-of-use are not as great a concern for the trained interviewer as for the naive interviewee if asked to use a computer to answer questions.
In CASI, computer-assisted methods help the respondent to complete a self-administered survey.
In A-CASI, the respondent uses headphones connected to a laptop computer to listen to questions that have been digitally recorded to an ACASI program, then keys his/her responses directly into the computer. This methodology has proven to be a highly successful means of gathering information on sensitive or personal topics, due to the added measure of privacy that results from the absence of an interviewer and the use of headphones.

32. CASI/A-CASI Advantages
Privacy
Branching in complex questionnaires
Automated consistency checks
Automated range checks
Automated adaptation of question wording based upon prior responses
Audio-CASI allows for standardized verbal delivery of all questions and does not require respondent literacy

33. Computer-Assisted Telephone Interview (CATI)
List samples of “live” residential phone lines are called to screen households and determine eligibility of members.
Once consent is obtained, interviewer uses a computer to guide them through asking survey questions, providing prompts, entering data, etc. as in A-CASI.
Interviewer also has some flexibility in dealing with respondent comprehension/literacy level.

34. CATI Advantages
Opportunity to access representative sample (costly with eligibility requirements that are less common).
Some advantages of live interviewer, but also anonymity of phone call.
Has capacities of computer-assisted technology with respect to branching, adaptation of wording, consistency/range checks, etc.

35. Telephone Audio Computer-Assisted Self-Interview (T-ACASI)
First a telephone interviewer contacts a household, screens for an eligible respondent, and secures consent.
Phone call then transferred to automated system where computer-controlled, pre-recorded questions are read aloud.
Respondent provides answers by pressing keys on a touch-tone phone.
At end, respondent can be returned to live person to close out interview.

36. Internet Surveys
Access to large sample at relatively low cost in relatively short time frame.
May provide access to difficult-to-reach populations.
As with other computer-assisted methods, may result in higher rates of reporting of sensitive information, and provides consistency and validity checks.
Rates of internet access are growing (if still disparities exist in different populations).
Participation rates of target population hard to gauge
Dropout rates
Direct data entry into database facilitates cleaning and analysis.

37. Summary: Survey Delivery Considerations
Mode of survey delivery clearly influence:
Respondent comfort and likelihood of self-disclosure (due to level of privacy)
Respondent ease of navigating through the survey if there are any skip patterns
Time and effort expended per respondent
The audience you can reach with your survey
Quality of data, data entry and analysis
While important, how you actually ask the questions is just as critical -- our next focus.

38. Survey Construction

39. Developing the Survey: Question Organization
Topics should be generally organized from least to most intimate
Initial questions are crucial -- respondent must get “hooked” by at least some questions which are felt to be personally salient
Initial questions must also require low effort (cognitive demands) to answer
Thus, if possible, avoid demographic questions to start (especially income!)

40. Developing the Survey: Question Organization
A survey -- no matter how delivered -- is a type of conversation.
Topics should follow an orderly sequence, with common questions grouped together.
This flow allows transitions to more intimate matters without jarring or acute awareness.
Transitional phrases can help in shifting set.
All this also provides respondent with notion that there is a rationale to questions asked.

41. Question Ordering
Be careful with filter questions: if not worded carefully, respondents will learn that a “no” response means avoiding subsequent questions.
Be careful about summary attitudinal questions: more specific queries to begin will influence summary responses by forcing respondent to think about their overall beliefs more carefully.
If possible, it is often helpful to ask a series of filter questions with yes/no responses, and then go back to the specifics that would follow from the appropriate responses.

42. Question Ordering
Having similar response choices reduces cognitive burden on respondent; grouping items with such options together is thus useful. It can also be useful to have the same number of options to reduce complexity (e.g., always using a four-point or five-point scale).
However, arrange types of questions to provide some variety and reduce response set.
Avoid asking unnecessary questions!!

43. Question Wording/Construction

44. Comprehension/Interpretation
Not just question of literacy, but influenced by cognitive burden and attention required
Wording choices
Length and complexity of survey items
Ordering of clauses
Consistency of phrasing
Avoid “not’s” or items whose meaning will be lost by inattention to a single word
Phrasing also influences response distribution along a Likert-type agree-disagree scale
Examples of Wording Revised in Items on Internalized Homophobia Scale (Michael Ross & Simon Rosser):
Avoidance of double negatives. Some participants requested clarification on these items:
3. It would not be easier in life to be heterosexual --> It would be harder in life to be homosexual
5. I do not feel confident about making an advance to another man --> Making an advance to another man is difficult for me.
8. I don’t like thinking about my homosexuality --> I avoid thinking about my homosexuality.
20. Most people don’t discriminate against homosexuals --> Only a few people discriminate against homosexual men.
23. I am not worried about other people finding out I am gay --> I am comfortable about other people finding out I am gay .
Preference for the use of homosexual/heterosexual as an adjective not a noun (as per modern usage).
4. Most of my friends are homosexual -> Most of my friends are gay/bisexual men
21. I feel comfortable about being homosexual -> I feel comfortable being a homosexual man.
Item 20 above.

45. Cultural Considerations
Individuals can come up with very different understandings of survey items, based upon their personal experiences.
The meanings that respondents bring to the survey experience can be strongly influenced by culture, acculturation and other background variables -- not just education and economic status.
These differences are highlighted in the vast literature that exists on different response patterns of different ethnic groups to the same standardized scales.
Pasick, Rena J., Stewart, Susan L., Bird, Joyce A., D'Onofrio, Carol N. (2001). Quality of Data in Multiethnic Health Surveys. Public Health Reports, 116 (suppl1):

46. Key Considerations
Examine all survey items with eye for their appropriateness for target population.
Be aware of pre-existing measures so as to not have to reinvent the wheel.
When using pre-existing scales, review in same way for clarity & ease of response.
Get feedback on item wording and construction from as many colleagues as possible.
Pre-test survey, using methods first described by Cannell et al. (1989).

47. Cognitive Burden & Recall Demand
Recall accuracy is influenced by a variety of factors, including:
salience/vividness of behavior,
time frame,
complexity of response information necessary (ever, counts, close-ended categories, etc.),
frequency and consistency of behavior,
effort respondent is willing to exert,
cueing techniques and prompts

48. Time Frame - Sexual/Drug Use Behaviors
3 month time frame generally considered the outside range of accurate reporting; some suggest limiting to 1 month.
Longer time frame may be possible if asking if a behavior ever happened.
One important timeframe consideration is dealing with infrequent behaviors -- what do you want to capture with respect to either behavior events or possible behavior change (if longitudinal study)?

49. Sensitivity
Item sensitivity not consistent across populations (e.g., sexual behavior among MSM vs. general population, drug use behavior among heavy drug abusers/IDUs, discussing sexuality among different ethnic groups).
Rationale for sensitive questions helps response rates.
Asking if “ever” occurred first, prior to asking about (for example) the last month, may be less charged for respondent.
Importance of neutral wording of question and (if IAQ) careful training of interviewer

50. In Closing ...
Selecting your methods of data collection must be carefully considered based upon target population, study questions and design, and costs.
No matter what the method of data collection, your instrument is critical.
Survey design is a combination of art and science, with the “art” strongly reliant upon interviewing skills.
The science of survey design currently has a wealth of research literature to examine and guide you.