General Online Research Cologne, 6 th of March 2014 Holger Lütters – Hochschule für Technik und Wirtschaft Berlin Michael Kraus – Hochschule für Wirtschaft und Recht Berlin Daniel Westphal – Hochschule für Technik und Wirtschaft Berlin Moving answers with the Gyroscale: Using the mobile device’s gyroscope for market research purposes

Mobile Market Research Technology development vs. research approaches 1

Lütters / Kraus/ Westphal· Moving answers with the Gyroscale · #GOR14 · Cologne 6 th of March Mobile Market Research is definitively happening… Situation of Mobile Market Research in 2014 : Use of mobile device is growing incredibly (> 1 billion smartphones in use) Technology limitations restrict research ideas Innovation in mobile market research: black/white questionnaires are formatted to fit on the smallest colored screens Lack of ideas how to incorporate the power of mobile devices into the world of online research Our contribution to the development of new interaction forms: The Gyroscale A new form of interaction design in the interview process using a mobile device’s gyroscope to capture interview answers

Lütters / Kraus/ Westphal· Moving answers with the Gyroscale · #GOR14 · Cologne 6 th of March … but other rules may apply in the mobile context Market Research does not thrill respondents Questionnaires tend to be boring for respondents in times of social media quizzes The growing amount of entertaining interactions like Facebook quizzes is catching the attention of respondents Willingness to participate in questionnaires steadily goes down … Mobile MR tends to make things worse (for researcher & respondent) Smaller screen size Less control of the surrounding Less tools for interaction available More time needed to complete Higher cost with Lack of specific methodology …

Lütters / Kraus/ Westphal· Moving answers with the Gyroscale · #GOR14 · Cologne 6 th of March Two major directions of mobile questionnaire development: Responsive Design vs. Device Agnostic Design Source: Example: Matrix question for desktop use Example: Identical question in responsive design on mobile device Responsive Design Adaption of questionnaire layout according to the respondent’s technology used Device Agnostic Design One version fits all design without necessary installation of specialized tools Source:

Development of the Gyroscale From idea to technology development 2

Lütters / Kraus/ Westphal· Moving answers with the Gyroscale · #GOR14 · Cologne 6 th of March Lütters/Westphal/Heublein at GOR 2007 (Leipzig): “The future of graphical scaling is interactive” Flash interaction with movement and sound Non-interactive scales with AJAX technology Interactive tests of the team based on Flash technology in 2007Examples based on Ajax technology

Lütters / Kraus/ Westphal· Moving answers with the Gyroscale · #GOR14 · Cologne 6 th of March Idea: Interaction through device movement: The Gyroscale literally moves answers The Gyroscale® is available in questfox® Source: Gyroscale® and questfox® are trademarks of pangea labs GmbH Switzerlandwww.pangealabs.com

Lütters / Kraus/ Westphal· Moving answers with the Gyroscale · #GOR14 · Cologne 6 th of March Demonstration of the Gyroscale in action Screencast recorded on a hybrid laptop with mouse input and touch functionality

A study of HTW Berlin with support from Panelbiz Berlin Setup and field report 3

Lütters / Kraus/ Westphal· Moving answers with the Gyroscale · #GOR14 · Cologne 6 th of March HTW Berlin study with identical content to three "sensitive" topics in order to induce deviant answer patterns in four technical routes Four device categories were targeted in comparison: 3 sensitive topics to provoke deviant answers: Device1 Desktop Device 2 Laptop Device 3 Tablet Device 4 Smartphone PoliticsSexDrug Abuse Target n=100Target n=200 Target n=200 Target n=100 Full report under

Lütters / Kraus/ Westphal· Moving answers with the Gyroscale · #GOR14 · Cologne 6 th of March minutes survey in Germany with sampling based on device used Field report Representative panel study with Panelbiz.com using SaaS Tool questfox.com First Response Monday 26 th of August 2013 Last Response Monday 1 st of September Contacts 662 complete Interviews Drop-out rate 8,5%-22,7% Language German Participation Time 10 minutes and 35 seconds (mean) Full report under

Lütters / Kraus/ Westphal· Moving answers with the Gyroscale · #GOR14 · Cologne 6 th of March Analytic Hierarchy Process Saaty as a methodical framework to use the pairwise answering technique Source: Goal Criterion 1Criterion 2Criterion 3Criterion 4 4 Elements = 6 pairwise comparisons with results about Ranking order Weighting of each criterion Consistency check for all 6 comparisons

Lütters / Kraus/ Westphal· Moving answers with the Gyroscale · #GOR14 · Cologne 6 th of March A background script senses the device‘s capabilities of using the gyroscope and switches to the appropriate version Question as shown on a Desktop/Laptop Question as shown on a Mobile Device (Gyroscope detected) Source: examples of standard slider scale and Gyroscale in questfox.com

Lütters / Kraus/ Westphal· Moving answers with the Gyroscale · #GOR14 · Cologne 6 th of March Gyroscale interventions in a complex questionnaire setting with 34 question pages 2 interventions with the Gyroscale in 34 questions 9 comparisons to be performed during interview Reactive asking about the device used while background tracking the functionality of a gyroscope in the mobile device Out of 437 mobile devices a gyroscope detection was positive on 359 devices (=82%) 6 comparisons 3 comparisons Gyroscale interventions:

Results of the first study using the Gyroscale Non-reactive results 4a

Lütters / Kraus/ Westphal· Moving answers with the Gyroscale · #GOR14 · Cologne 6 th of March Drop-out rates of mobile devices significantly higher Smartphone generates the highest drop-out rate Base: n=663 complete respondents (Completion Rate after Screenouts/Quota-Full rules of the panel) Drop-out rate in% of qualified starters after screening phase 22,7% 9,4% 8,5% 14,9% time in sec =100%+9%+23%+36% Completes (starters) n=107 (113)n=118 (123)n=209 (264)n=228 (330)

Lütters / Kraus/ Westphal· Moving answers with the Gyroscale · #GOR14 · Cologne 6 th of March Drop-out for mobile devices higher all through the questionnaire Gyroscale not responsible for higher drop-out rate Drop-out measurement after screening-phase (Question 11) 1st Gyroscale intervention 6 pairwise comparisons 2nd Gyroscale intervention 3 pairwise comparisons

Results of the first study using the Gyroscale Reactive results 4b4b

Lütters / Kraus/ Westphal· Moving answers with the Gyroscale · #GOR14 · Cologne 6 th of March pairwise comparisons end up in an identical ranking of items independently from the device used n=107 n=117 n=204 n=211 6 pairwise comparisons with Analytic Hierarchy Process using different technologies Input with Mouse Touchpad Trackball etc. Input with device movement using Gyroscale

Lütters / Kraus/ Westphal· Moving answers with the Gyroscale · #GOR14 · Cologne 6 th of March pairwise comparisons showing only minor differences in the data from the 4 different devices n=107 n=117 n=204 n=211 3 pairwise comparisons with Analytic Hierarchy Process using different technologies Input with Mouse Touchpad Trackball etc. Input with device movement using Gyroscale

Results of the first study using the Gyroscale Consistency check 4c

Lütters / Kraus/ Westphal· Moving answers with the Gyroscale · #GOR14 · Cologne 6 th of March Desktop generates the worst consistency values … it is today’s industry standard of measuring AHP consistency 1. Rating: 0, Rating: 0, Rating: 0, Rating: 0, Rating: 0, Rating: 0, Rating: 0, Rating: 0,266 Base n = 662; consistency measurement in the Analytic Hierarchy Process, No repetition of inconsistent pairwise comparisons; Consistency was measured in two different AHP Decision Trees (1. rating, 2. rating) without warm-up The higher the value, the more inconsistent the response pattern; Classic limit after Thomas L. Saaty: CR <0,1 n=107n=118 n=209 n=218

Lütters / Kraus/ Westphal· Moving answers with the Gyroscale · #GOR14 · Cologne 6 th of March PCLaptop Tablet (Gyroscale) Phone (Gyroscale) very consistent16,80%22,90%26,30%23,20% consistent23,40%22,00%20,60%18,90% less consistent47,40%44,10%34,90%38,20% inconsistent11,20%6,80%10,00%11,00% extremely inconsistent0,90%0,00%1,40%0,90% Basen=107n=118n=209n= Rating0,272 0,236 0,219 0, Rating0,281 0,236 0,253 0,266 The quality indicator “consistency” is surprisingly positive on both mobile devices Base n = 662; consistency measurement in the Analytic Hierarchy Process, No repetition of inconsistent pairwise comparisons; Consistency was measured in two different AHP Decision Trees (1. rating, 2. rating) without warm-up The higher the value, the more inconsistent the response pattern; Classic limit after Thomas L. Saaty: CR <0,1;

Limitations of the Gyroscale A critical reflection on the opportunities of the Gyroscale for market research studies 5

Lütters / Kraus/ Westphal· Moving answers with the Gyroscale · #GOR14 · Cologne 6 th of March The Gyroscale offers some divertissement during a questionnaire, but needs to be substantiated with a specific methodology Gyroscale Pros Measurement with entertaining character Impulsive measurement with movements might be more spontaneous (does not need to be worse than answers with a mediated interface (mouse; touchpad etc.) Very suitable for pairwise comparisons like Semantic Differential, Analytic Hierarchy Process, Conjoint Analysis… Gyroscale Cons 82% detection rate of gyroscope functionality is not representative (like Flash in 2004) Limited to one answer per screen Technology issues between iOS, Windows Phone, Android etc. Every device needs to be programmed once and controlled regularly Device size has an influence on the reaction (angle) and might cause different effects Hybrid device with touch and mouse Gyroscale might also become boring once it is integrated everywhere

Lütters / Kraus/ Westphal· Moving answers with the Gyroscale · #GOR14 · Cologne 6 th of March Mobile Research: The issue of interaction design seems to be more significant than the mobile usage itself Response patterns of answers to different devices need to be looked at in general. Surprising difference in the response patterns between the categories laptop and desktop New problems for researchers of perception and attention in the mobile arena Device agnostic is the development direction. Proprietary developments (Apps, Flash, Silverlight etc.) bring us back to the stone ages of computer development The future of research will remain mobile: Maybe device intermediated answering is part of the past already

Lütters / Kraus/ Westphal· Moving answers with the Gyroscale · #GOR14 · Cologne 6 th of March Our next study will compare data of mobile surveys only Who would like to support that approach? GyroTouch NoiseVoice

Lütters / Kraus/ Westphal· Moving answers with the Gyroscale · #GOR14 · Cologne 6 th of March Minute market research seems to be possible Gyroscale in combination with decision making models Quick answers to complex models Complexity of the model distributed in a network of users Quick surveys with serious background Comparable to the 7 Minute Abs video from “Something about Mary”

Lütters / Kraus/ Westphal· Moving answers with the Gyroscale · #GOR14 · Cologne 6 th of March Thank you for your interest in the Gyroscale Test it with your mobile device under Authors: Holger Lütters – HTW Berlin Michael Alexander Kraus – HWR Berlin Daniel Westphal – HTW Berlin Contact information Prof. Dr. Holger Lütters Hochschule für Technik und Wirtschaft Berlin Fachbereich 3 Treskowallee Berlin Raum TA VG xing.com/profile/Holger_Luetters de.linkedin.com/in/luetters/ Thanks to the supporters of the study: Gyroscale® is a development of pangea labs GmbH Switzerland and is available in the SaaS survey tool questfox® Special thanks to Benjamin Schubert for design input