1. The basics of Social Science Research Lecture 3
Dr. Jordy Gosselt
Department of Communication Science

2. Today Research methods and Data collection
Making a questionnaire
April 3th: Data analysis (making a data file in spss, calculating the reliability of your instrument, making variables, calculating Means/SD, comparing groups)
Crime Science Theories (M. Junger)
 Book Chapter 5, 6, 7, 9, 10
[Bha2012] Bhattacherjee, A. (2012). Social Science Research. Principles, Methods, and Practices. Tampa, Florida: University of South Florida. PDF

3. Now: Making a questionnaire
Social Science Research Training: Aim of the study?

4. Now: Making a questionnaire
Study: The privacy paradox observed in normal behavior
Theory:
Users claim to be highly concerned about their privacy (=attitude)
Consumer’s choice to download an app is driven by (1) popularity; (2) usability; (3) price, without much hesitation (=actual behavior)
Privacy paradox?
For the largest part of lay people, technical processes that run in the background are not understandable: users make use of risky online services even if they have privacy or security concerns.
Knowledge gap?
Users have the tendency to accept all kinds of permissions although having privacy concerns.
RQ1: To what extent do people understand how permissions work?
RQ2: What are the perceived concerns regarding permissions?
RQ3: To what extent do people behave differently regarding different kinds of permissions and does the type of app have an influence?
RQ4: To what extent is there a gap between self-reported and actual behavior?

5. Now: Making a questionnaire
Study: The privacy paradox observed in normal behavior
RQ1: To what extent do people understand how permissions work?
RQ2: What are the perceived concerns regarding permissions?
RQ3: To what extent do people behave differently regarding different kinds of permissions and does the type of app have an influence?
RQ4: To what extent is there a gap between self-reported and actual behavior?

6. Now: Making a questionnaire
Combination of survey, qualitative and experiment
Part 1: Evaluation of participants’ general smartphone and app usage: survey
Part 2: Measuring users’ general privacy sensitivity: open question and survey
Part 3: Level of comfort with accepting permissions and granting access: experiment
Part 4: Comprehension of permissions: quiz
Part 5: Permissions and perceived concern in different risk scenarios: survey
RQ1: To what extent do people understand how permissions work? (Part 4)
RQ2: What are the perceived concerns regarding permissions? (Part 2 and Part 5)
RQ3: To what extent do people behave differently regarding different kinds of permissions and does the type of app have an influence? (Part 3)
RQ4: To what extent is there a gap between self-reported and actual behavior? (Part 1 and Part 3)

7. Now: Making a questionnaire
Participant information (goal / relevance / motivating)
Informed consent (withdrawal)
Ask for demographical data:
Participant information + informed consent = see CCS website
Demographics: Ask as much as you need to carefully describe your sample but do not ask useless information

8. Now: Making a questionnaire Part 1: Evaluation of participants’ general smartphone and app usage : survey
Measuring general smartphone and app usage  is there any influence of (indicated) behavior on risk perception

9. Now: Making a questionnaire Part 1: Evaluation of participants’ general smartphone and app usage : survey
Measuring general smartphone and app usage  is there any influence of (indicated) behavior on risk perception

10. Now: Making a questionnaire Part 1: Evaluation of participants’ general smartphone and app usage : survey
Measuring general smartphone and app usage  is there any influence of (indicated) behavior on risk perception

11. Now: Making a questionnaire Part 2: Measuring users’ general privacy sensitivity: open question and survey

12. Now: Making a questionnaire Part 2: Measuring users’ general privacy sensitivity: open question and survey
Scale: Westin Privacy Index (3 items, privacy sensitivity; see Kumaraguru & Cranor, 2005)
Westin Privacy Index = General privacy sensitivity
Does the general privacy sensitivity has an influence on risk perception and level of comfort when accepting permissions?
[Kum2005] Kumaraguru, P. & Cranor L. F., Privacy indexes : a survey of Westin's studies. Technical report. Carnegie Mellon University: Pittsburgh, Pennsylvania.

13. Now: Making a questionnaire Part 3: Level of comfort with accepting permissions and granting access: experiment
Level of comfort and granting access for 2 apps:
Hedonic app (tetris) vs utilitarian app (to-do-list)
Choice of apps based on a pre-test
Selection of permissions (n=7) based on Taylor & Martinovic, 2016
Answers on comfort and granting access
Is there a difference in level of comfort between the evaluation and allowing of permissions of hedonic vs utilitarian app?
[Tay2016] Taylor, V.F., & Martinovic, I. (2016). SecuRank: Starving permission-hungry apps using contextual permission analysis. Proceedings of the 6th Workshop on Security and Privacy in Smartphones and Mobile Devices, pp , October , 2016 Vienna, Austria.

14. Now: Making a questionnaire Part 4: Comprehension of permissions: quiz Adopted from Felt et al. (2012)
What is your comprehension level of permissions?
Does the comprehension level match with risk perception and level of comfort when granting access to permissions?
Adopted from Felt, Ha, Egelman, Haney, Chin & Wagner, – Permission Comprehension Quiz to the permissions asked in the experiment. Approx. 50% of the items are reused, the rest is modified.
[Fel2012] Felt, A. P., Ha, E., Egelman, S., Haney, A., Chin, E., & Wagner, D. (2012). Android permissions: User attention, comprehension, and behavior. Proceedings of the eighth symposium on usable privacy and security,

15. Now: Making a questionnaire Part 5: Permissions and perceived concern in different risk scenarios: survey
Ranked list of risk scenarios: Is there a match between level of comfort and granting access to permissions?

16. Next: Analyzing results
Combination of survey, qualitative and experiment
Part 1: Evaluation of participants’ general smartphone and app usage: survey
Part 2: Measuring users’ general privacy sensitivity: open question and survey
Part 3: Level of comfort with accepting permissions and granting access: experiment
Part 4: Comprehension of permissions: quiz
Part 5: Permissions and perceived concern in different risk scenarios: survey
RQ1: To what extent do people understand how permissions work? (Part 4)
RQ2: What are the perceived concerns regarding permissions? (Part 2 and Part 5)
RQ3: To what extent do people behave differently regarding different kinds of permissions and does the type of app have an influence? (Part 3)
RQ4: To what extent is there a gap between self-reported and actual behavior? (Part 1 and Part 3)

17. To sum up Combination of research methods
Method dependent on RQ (knowledge, attitude, behavior)
Use existing scales (reliable!)
Do a pre-test if possible (validity!) regarding:
Stimulus material
Questions + answers

18. Data Collection Type of data collection: qualitative vs. quantitative
Expected amount of collected data
Online questionnaire: Between 200 and 250 respondents
Hard-copy questionnaire: Between 100 and 150 respondents
Experimental designs: Depending on number of conditions (min 30 resp. per condition)
Interviews: Between 15 and 20 participants
Focus groups: Between 4 and 6 group sessions
Content analysis: Depends on the research design
Observations: Depends on the research design

19. Data Collection: Qualtrics
Data collection procedure
Online survey tool “Qualtrics” is available in cases of quantitative data collection (i.e. surveys, experiments, sorting tasks, heat maps...)
Easy to use and distribute
Direct export to SPSS

20. Data Collection: Qualtrics
Data collection procedure
Online survey tool “Qualtrics” is available in cases of quantitative data collection (i.e. surveys, experiments, sorting tasks, heat maps...)
Easy to use and distribute
Direct export to SPSS

21. Your research proposal:
Refine RQ(s) based on literature
De/Refine your concept(s), construct(s) and variable(s)
Determine how to examine your RQ(s)
The research method must fit your RQ(s)
Define your target group and sampling method
Operationalize your chosen method
Make or find scales / make experimental groups / make interview questions / etc. and think of validity and reliability
Fill in the documents for the ethical committee
Contact your student assistant if you have questions