UEM College 9 mei, 2006 The Paradox of the Assisted User: Guidance can be Counterproductive Christof van Nimwegen

UEM College 9 mei, 2006 Having a plan…..the basis of success? O’Hara & Payne:  Plan-based:Use strategies from long term memory  Display based:Rely on interface information Making an interface more costly provoked - More plan-based strategies - Better transfer to same domain problems Zhang 1997: Externalizing information can be useful for cognitive tasks; the more information is externalized, the easier it is to solve the problem

UEM College 9 mei, 2006 Background: Externalization vs. Internalization  Important for interaction design how much information should be available on the interface (externalization) as opposed to having users encode and store this information in memory (internalization).  We investigate conditions whereby externalizing the information in the interface, or requiring it to be internalized respectively, enhances task performance of users.  This research contributes to formulate guidelines for GUI design in the context of problem solving tasks.

UEM College 9 mei, 2006 Externalization vs. Internalization: Example Externalization  Representation with relevant feedback  Remembering certain task-related knowledge is unnecessary  This relieves working memory (low cognitive load) Internalization No support: Task-related information is unavailable on the interface It needs to be internalized (high cognitive load) and stored in memory Knowledge in memory of user is more solid and detailed (recall, not recognition)

UEM College 9 mei, 2006 General research questions  In literature, there is no real agreement, but externalization in modern GUI’s is more or less default  But does this “support” of externalization …  really result in better “performance” after all?  make a task more pleasant in the long run?  help when “learning something” is the aim? (think of software in education)  with transfer tasks?  result in better knowledge of the underlying task?  what about interruptions?

UEM College 9 mei, 2006 Earlier experiment: Problem choice Classical: Missionaries & Cannibals Bring all of them to the other side Rules: - Maximum 3 in boat - Minimum to sail is 1 (not empty) - Cannibals never in the majority or they will eat the missionaries Abstract version: Ball & Boxes Transfer: Realistic M&C

UEM College 9 mei, 2006 Path

UEM College 9 mei, 2006 Material first experiment (with greyed-out items)  Abstract puzzle: variation on Missionaries & Cannibals INT EXT  A more realistic transfer task: Missionaries & Cannibals

UEM College 9 mei, 2006 Results experiment 1 Internalization resulted in  better problem knowledge afterwards  better performance after long interval  better performance on transfer task after interval Externalization had no advantages at all!

UEM College 9 mei, 2006 Externalization and planning? Externalization > Relying on interface discourages planning > Shallow thinking > Worse performance Internalization > Stimulates planning > More engagement > Better performance

UEM College 9 mei, 2006 Second experiment: more realistic task Independent variables  Interface style (INT-EXT) We used more realistic material. Instead of abstract puzzles a conference planning task  Cognitive style Cognitive style We included an attitude trait. Before we tried to externally induce some state of mind, now we want to look at stable trait in people. We chose for the “Need for Cognition” (NFC). NFC is a well proven construct measuring the tendency of individuals to engage in and enjoy effortful cognitive tasks

UEM College 9 mei, 2006 Hypotheses  H1: Internalization leads to a more plan-based strategy and better performance than externalization. Having information externalized tempts users not to form plans and rely on the interface. The internalized condition lacks this guidance and is encouraged to plan and think before acting.  H2: High NFC leads to a more plan-based strategy than low NFC. People with high NFC have high intrinsic motivation to think and engage in effortful cognitive tasks, will show more plan-based behavior, and perform better

UEM College 9 mei, 2006 Method Subjects  43 subjects, 17 (40%) male, 26 (60%) female.  Aged 19 to 32 years  Following or recently having followed higher education  The experiment took at most one hour, 5 euro reward Design 2*2 Independent variables:Interface style(INT <> EXT) Cognitive style(HIGH <> LOW) (median split) Dependent variables:Various performance measures (times, moves) Strategy Knowledge & Opinions

UEM College 9 mei, 2006 The application  The “Conference Planner” application simulated the planning of speakers (it logged all the moves)  Each of the speakers had different demands. They had to be scheduled in one of three conference rooms  The conference rooms each had its own facilities and availability  Without planning, the scheduling will not be optimal and extra moves besides the optimal path must be made  The difference between EXT and INT was implemented by providing information concerning the legality of moves/actions

UEM College 9 mei, 2006 Conference planner: externalization and internalization  In EXTERNALIZATION it showed where speakers can be placed.  Slots in the timetable (satisfying constraints being available) turn green  Not the best slot, just legal ones  In INTERNALIZATION the interface, this feedback was absent  Subjects need to look more at the constraints and think and plan themselves

UEM College 9 mei, 2006 Measures…..

UEM College 9 mei, 2006 Procedure 1. NFC 2. Conference Planner a) Trial 1 b) Trial 2 c) Trial 3 d) Trial 4 e) Trial 5 3. Post Experimental Questionnaire

UEM College 9 mei, 2006 Results: Time based measures (av. per 5 trails) Time before first move INT took longer than EXT (p<.05) INT:M=19.8, SD=7.4 EXT:M=15.3, SD=7.2 Time between moves INT took longer than EXT (p<.05) INT: M=4.8, SD=1.4 EXT: M=3.9, SD=1.3 Total time: no significant difference

UEM College 9 mei, 2006 Results: Move based measures (av. per 5 trails) Superfluous moves: INT made fewer than EXT (p<0.05) INT: M=2.5, SD=0.6 EXT: M= 4.3, SD=0.6 Reconsidered moves: INT made fewer than EXT (p<0.01) INT: M=0.5, SD=0.4 EXT: M=2.1, SD=0.4 Invalid moves : no significant difference

UEM College 9 mei, 2006 Strategy & opinion  Internalization subjects used the ‘most constraints first’ strategy more often. Tendency, (F(1,39)=3.21, p=0.08). INT: 2.4 times out of 5 (SD=1.76) EXT: 1.5 times out of 5 (SD=1.5)  “I sometimes did not know how to proceed with the arrangement of the speakers”, (F(1,39)=5.91, p<0.02). EXT:M= 5.24, SD =1.61 INT:M= 6.23, SD=0.97) (the item was scored reversely)

UEM College 9 mei, 2006 Conclusion Hypothesis accepted: The interface requiring more internalization resulted in  longer thinking time before subjects started  more time between moves  fewer superfluous moves and deviations, thus with greater efficiency This indicates that when information has to be internalized, more planning, active thinking and contemplation from the users is provoked

UEM College 9 mei, 2006 Conclusions H2: rejected. Cognitive style along the dimensions of NFC had no influence. It was Interface style again that predicted problem solving behavior quite strongly, and could overrule possible effects of pre-existing individual differences. Also, our subjects were volunteering university students, where the NFC average was very high.

UEM College 9 mei, 2006 Conclusions  If plan-based behavior is preferred, designers should be careful in giving users too much assistance (what is the goal?)  With multimedia being present in all corners of society, our findings can be valuable in the development of applications in the realm of education, multimedia learning or game based learning

UEM College 9 mei, 2006 Further research Understanding reactions to interface information (based on cognitive findings) is important in tuning software, thereby allowing it to achieve its goal. We will continue the research, and broaden the types of problem solving activities, and include other variables related to attitudes to problem solving.

UEM College 9 mei, 2006 Future research Severe Interruptions (PC Crash) Transfer Task (same task, different feeling) Use eye tracking (distraction? Where are they really looking?)

UEM College 9 mei, 2006 Deel 2 Wat jullie gaan doen!? Deel 2

UEM College 9 mei, 2006 Week 20 Maandag Werkgroep Usability lab (CGN,C 201) Groep 1: 13:00 Groep 2: 13:30 4 proefpersonen Groep 3: 14:00 4 proefpersonen Groep 4: 14:30 6 proefpersonen _____________ 14 pp totaal

UEM College 9 mei, 2006 Dinsdag - donderdag Experimentatie  2 personen aanwezig steeds  minuten per proefpersoon  Jullie krijgen een tekst protocol  Just drinks Nieuw in dit experimentje: LogsqareLogsqare wordt gebruikt voor video capture, in activity-maps Procedure 1. De NFC-lijst 2. De planningstaken (korte versie experiment (3 taken) 3. Questionnaire (drastisch ingekort, alleen opinions, op papier)

UEM College 9 mei, 2006 Data (stap 1) 1. CSV-file van NFC scoresNFC 2. CSV-file van taken (raw data)taken 3. Ingevulde korte (papieren) eind-vragenlijst 4. Videos en met activity chart afbeelding

UEM College 9 mei, 2006 Data (stap 2) Runnen macros Raw data > preciezere datapreciezere data Fill up the gaps> mbv. videos Preciezere data> SPSS

UEM College 9 mei,

UEM College 9 mei, 2006 Data (stap 3) & verwerking  Voeg data NFC scores en Vragenlijst toe aan SPSS file  Jullie krijgen van mij nog instructies tzt. Welke variabelen ik erin wil hebben  Stuur ALLE databestanden naar mij per mail  Ik krijg dus per proefpersoon:  Nfc-csv  PET-csv  Pet-csv.xls (preciezer)  Pet csv.xls.final(nog preciezer,no gaps)  SPSS-bestand compleet(alles!)  Alleen de videos en activity charts kunnen op de Usabilitylab PC blijven staan. Zet dit duidelijk in en folder met groepnummer, daarin mapjes met proefpersoon nummer/naam

UEM College 9 mei, 2006 En dan….analyse!  Jullie krijgen dan zo snel mogelijk van mij een samengevoegd spss-bestand terug (19-20 mei)  26 mei, 23:59 (een week later) is de deadline voor het groeps rapport  Wees precies!!  Je kunt tussendoor altijd mailen als er een probleem is SUCCES!!