1. Human Factor Evaluation for Marine Education by using Neuroscience Tools N. Νikitakos D. Papachristos Professor Ph.d. candidate Dept. Shipping, Trade and Transport University of Aegean MASSEP 2013 May 2013

2. www.stt.aegean.gr Human Factors Evaluation Research Methodology Case Study CONTENTS

3. www.stt.aegean.gr HUMAN FACTORS EVALUATION

4. www.stt.aegean.gr Maritime education user’s satisfaction objective criteria satisfaction phenomena Human Factor Evaluation (1)

5. www.stt.aegean.gr mixed approach to Human Factor evaluation ship’s bridge equipment usability and educational evaluation ship bridge interactive systems neuroscience tools of gaze tracking & speech recording for measuring emotional user responses Usability testing Human Factor Evaluation (2)

6. www.stt.aegean.gr Human Factors Evaluation (3) Neuroscience tools

7. www.stt.aegean.gr Human Factors evaluation ship manipulation systems design (interactive technologies) ergonomic few applications in industry cognitive ergonomics Human Factor Evaluation (4)

8. www.stt.aegean.gr Usability has been defined by ISO 9241 as “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use” Human Factor Evaluation (5)

9. www.stt.aegean.gr Effectiveness means accuracy and completeness with which users achieve specified goals Efficiency means resources expended in relation to the accuracy and completeness with which users achieve goals Satisfaction means freedom from discomfort and positive attitudes towards the use on the product Human Factor Evaluation (6)

10. www.stt.aegean.gr RESEARCH METHODOLOGY

11. www.stt.aegean.gr Defining satisfaction concerns the following parameters, which are being investigated: concerning software and educational scenarios, system usability as far as the system per se is concerned (total functionality), as well as the individual training and technical characteristics that complete the teaching act. Research Methodology (1)

12. www.stt.aegean.gr The research questions (RQ) set by the suggested research framework as follows: RQ-1: There is a relationship between the user’s optical attention to the user’s satisfaction either for the software or the scenario? RQ-2: There exists a relationship between the training characteristics, user satisfactions and by extension optical attention? Research Methodology (2)

13. www.stt.aegean.gr The suggested research method aims at interpreting, determining and evaluating the data of the biometric tool in combination with the conventional methods (qualitative, quantitative) results based on the factors (relationships) that is possible to influence the user’s satisfaction Research Methodology (3)

14. www.stt.aegean.gr Research Methodology (4) Measurements software scenario Quantitative data (questionnaires) Qualitative data (interview) Relationships between parameters-factors Interpretation procedure

15. www.stt.aegean.gr CASE STUDY

16. www.stt.aegean.gr The case study aims the following: the evaluation of the user satisfaction from using the ECDIS software and scenario and educational evaluation of ECDIS from the user’s point of view (opinions). Case study (1)

17. www.stt.aegean.gr first (random) sampling (January 2012 until May 2012), in the Information Technologies Laboratory of the National Marine Training Centre of Piraeus 31 Marine officers video recording of ~23 minutes per student Case study (2)

18. www.stt.aegean.gr Case study (3) ECDIS operation ECSIS Lab room Eye tracker “Face Analysis“ software - station experiment

19. www.stt.aegean.gr Stage 1: Information about the experiment, Presentation of the acceptance document by the user-trainee (estimated time duration 5 - 10 minutes) Stage 2: Completion of a user’s profile and of the assessment survey concerning educational and technical characteristics (questionnaire, T3) by the trainee (estimated time period 10 - 15 minutes) Stage 3: Equipment installation (gaze tracking device) and configuring the parameters (T1) Stage 4: Video recording (T1) in combination with filling in a work sheet (T3) by the researcher (estimated time duration 20-25 minutes) Stage 5. Completion of the process (device disconnect) through a semi structured interview & questionnaire (T2, T3, T4) with the user (estimated at 5 – 10 minutes) Case study (4)

20. www.stt.aegean.gr Tool-1 (T1): the optical data registration will be conducted by the “Face Analysis” software in connection with a Web camera set on the computer in which there is the subject of the research (ECDIS) Tool-2 (T2): Use of a microphone for voice recording (interview) Tool-3(T3):Questionnaires using for opinion/attitudes/expectation/self-evaluation and observing Tool-4 (T4): Usability assessment tool SUS Case study (5)

21. www.stt.aegean.gr Case study (6) Eye gaze vector Schedule of eyes & head pose Distance from monitor Head roll (angle), HR XoXo Eyes Quality parameter (eye gaze trucking) values (horizontal)>0: mean out of screen values (vertical)→-18 view of the center of the screen ~0 attention in screen ~1 & >1 no attention >1 close to the screen <1 away from the screen Values >10 o degrees, (high mobility) Values <10 o degrees (attention depending on the scenario EL HR= --------- HL Horizontal Level, HL Eye Level, EL Βiometric tool parameters interpretation (‘Face Analysis’)

22. www.stt.aegean.gr The data of experiment come from three sources (by using SPSS, Excel): questionnaires, SUS Tool, optical data (gaze tracking) and Interviews (voice recording) Case study (7)

23. www.stt.aegean.gr The results are shows: a relationship between Gaze parameter and Usability assessment of users. The gaze parameter depending from SUS score. It shows attention increases as assessment from ECDIS software (RQ-1), a relationship between SUS score and training characteristics (total assessment, time schedule) and a strong relationship between ECDIS satisfaction and Scenario Satisfaction (RQ-2). It seems the scenario operation depending from software environment (navigation, interface), High usability for ECDIS software (questionnaires evaluation, SUS tool) and high score for Training program evaluation (National Marine Training Centre of Piraeus). Case study (8)

24. www.stt.aegean.gr Case study (9) Marine OfficersMale (29) Female (2) Age’s scale 24-35 years16 (51.6 %)2 (6.45%) 36-45 years6 (19.35%)0 >45 years7 (22.5%)0 Officer order A’13 (41.9%)0 B’4 (12.9%)0 C’12 (38.7%)2(6.45%) Eye diseases10 (32.2%)0 Eye Operation1 (3.2%)0 Sample’s structure

25. www.stt.aegean.gr Case study (10) Marine OfficersTraining ProgramECDIS Total AssessmentTime ScheduleEducational GoalsNavigationInterfaceMultimedia Evaluation scale Very high15 (48.3 %)18 (58.06%)19 (61.25%)15 (51.6 %)12 (38.7%) High7 (22.6%) 1 (3.22%)9(19.35%)13 (41.9%)16 (51.6 %) Medium6 (19.35%)4 (12.9%)2 (6.45%)7 (19.35%)6 (19.35%)3 (9.6%) Low3 (9.6%)2 (6.45%)0000 Total31 (100%) ECDIS – Training program Evaluation

26. www.stt.aegean.gr Case study (11) NoVariable correlatedSpearman’s rhoSig. (2-tailed)Remark 1SUS score – ECDIS Satisfaction.453.011Positive 2SUS score - Interface.469.008Positive 3SUS Score - Gaze.393.029Positive 4ECDIS satisfaction – Total Assessment.398.026Positive 5ECDIS satisfaction – Time Schedule.468.008Positive 6ECDIS satisfaction – Navigation.531.002Positive 7ECDIS satisfaction – Interface.563.001Positive 8ECDIS satisfaction – Scenario Satisfaction.708.000Positive 9Scenario Satisfaction - NAvigation.373.039Positive Correlations between variables of research tools

27. Thank you nnik@aegean.gr