1. THE EFFECT OF TIME PRESSURE IN CONSTRUCTION SIMULATIONS ON STUDENTS‘ LEARNING PROCESS Martin Jaeger Australian College of Kuwait, Kuwait Desmond Adair University of Tasmania, Australia INTED 2013 - Valencia (Spain) 4.-6. March 2013 INTED 2013

2.  Safety education for engineering students can be fairly unreal in that it focuses exclusively on safety issues.  For engineers in real world situations, safety issues can be side issues interwoven into other activities, which are usually given priority.  This leads to ill-structured and complex workplace problems, which has implications for engineering education in that students should be required to learn how to reconcile and manage conflicting constraints [3].  The authors of this paper propose to supplement safety education with simulations of real world construction situations, which include social aspects such as communication with other project members, tasks not directly related to safety issues, and time pressure.  However, a question arises, ‘How do these distractions influence the learning process of construction safety’? INTED 2013 - Valencia (Spain) 4.-6. March 2013 Introduction

3. INTED 2013 - Valencia (Spain) 4.-6. March 2013 Background  When simulations aim at preparing students for stressful situations such as situations under time pressure, stress exposure training becomes necessary [4].  Simulations which reproduced real-world situations have been shown to lead to good outcomes [7, 8, 9].  At the same time, it was suggested to include within a simulation only learning processes, such as reflection, for the development of cognitive skills [10].  Literature includes substantial evidence for time pressure leading to more heuristic processing, but, the literature also shows that this does not necessarily mean worse performance [11].  Furthermore, Goodie and Crooks [11] proved experimentally that time pressure may lead to an improved performance.

4. INTED 2013 - Valencia (Spain) 4.-6. March 2013 Questions to be answered 1. Are there significant differences in students’ learning process between usages of simulations of construction sites with time pressure, versus, the same simulations without time pressure? 2. Are there significant relationships between influential variables and the students’ learning process between simulations with time pressure, versus, simulations without time pressure? Influential variables here include motivation to derive acceptable reactions, perception of simulator’s ease of use, and the perception of the necessity to concentrate. 3. Do these influential variables influence the two student groups differently comparing simulations with time pressure, versus, simulations without time pressure?

5. INTED 2013 - Valencia (Spain) 4.-6. March 2013 Method - Design  The study is based on a quasi-experimental post-test design, which compares the learning process of an experimental group (i.e. a group of students exposed to a simulator with time pressure), with the learning process of a control group (i.e. a group of students exposed to the same simulator, but without time pressure).  The following slide shows an example of the simulator screen.  The time pressure was generated by the count-down of a given time limit and re-occurring on-screen messages which commanded the student to speed up.

6. INTED 2013 - Valencia (Spain) 4.-6. March 2013 Method - Simulator

7. INTED 2013 - Valencia (Spain) 4.-6. March 2013 Method - Sample  The students were in the second semester of their second year of a university course.  The sample comprised 58 students enrolled in engineering courses and they were assigned to the experimental and the control group.  Three students dropped out which led to a response rate of 95%.  The average age of the sample was 21.5 years and, 58% of the students were male students.

8. INTED 2013 - Valencia (Spain) 4.-6. March 2013 Method – Experimental Situation  Both groups were exposed to an in-house simulator developed using the JAVA programming language.  The scenarios were identical and included personalized “real-world” questions which students were asked to answer based on displayed images taken from OSHA [14].  Construction safety aspects were included by three multiple-choice questions, introduced with, “By the way, what else would you do looking at the displayed situation?”  Finally, both groups of students were required to complete a questionnaire in order to self-evaluate their perceived influence of influential variables.  Both groups were exposed to the same lecture on construction safety one week prior to the experiment

9. INTED 2013 - Valencia (Spain) 4.-6. March 2013 Method – Variables  Following an indirect approach, the learning process was measured here by the simulator recording the total learning time (in seconds, variable name: total time) a student needed to work through the 16 scenarios and the number of attempts (variable name: total attempts) a student needed in order to come up with answers which were identified by the simulator as acceptable.  Following the approach of Ponzurick et al. [16], who justified measuring students’ satisfaction through a single item, it was decided to measure students’ perception of the influence of influential variables (independent variables) as single-item measures.

10. INTED 2013 - Valencia (Spain) 4.-6. March 2013 Method – Variables  The influential variables were measured as follows: Influential variableVariable nameScale Motivation to give correct answersMotivation5 point response scale Simulator ease of useEase of useYes, No Students’ perception of necessary concentration Concentration4 point response scale  Students’ general interest in construction management (interest) and their gender (gender) were recorded in order to identify potential differences between the experimental and the control group. The general interest was measured based on a 5 point response scale (1 = very low, 5 = very high).

11. INTED 2013 - Valencia (Spain) 4.-6. March 2013 Method – Analysis  The two pre-test variables (interest, gender) were analysed applying an analysis of variance (ANOVA). Based on a confidence interval of p = 0.95, no significant differences were found.  An ANOVA was applied to compare the total attempts to reach acceptable answers and the total time students needed between the two student groups (Table below). Simulator with time pressure Simulator without time pressure meanSDNmeanSDNFF crit Attempts222.2328205.87270.5734.023 Time [seconds] 38737.432837032.14273.184.023

12. INTED 2013 - Valencia (Spain) 4.-6. March 2013 Method – Analysis  The second research question was aimed at exploring the relationships between the influential variables (motivation, ease of use, concentration) and the total number of attempts to work through the scenarios as well as the total time to work through the scenarios for both simulator types. The table below shows the correlations between the influential variables and the total number of attempts. The correlations of the influential variables with the total number of attempts are for both simulator types insignificant. with time pressureno time pressure Motivation-0.09-0.07 [28][27] Ease of use0.050.00 [28][27] Concentration0.31 [28][27]

13. INTED 2013 - Valencia (Spain) 4.-6. March 2013 Method – Analysis  The table below shows the correlation between influential variables and the total time for both simulator types. The students’ motivation to score high correlates higher with the total time needed for the simulator with time pressure, and the correlation of the perceived necessary concentration correlates negatively for the simulator with time pressure. Correlation of the perceived ease of use is the same for both simulator versions. with time pressureno time pressure Motivation0.200.10 [28][27] Ease of use0.03 [28][27] Concentration-0.190.26 [28][27]

14. INTED 2013 - Valencia (Spain) 4.-6. March 2013 Method – Analysis  The third research question explores if there is any difference of influence on the two student groups. The table below contains the results of an ANOVA and shows that there is no significant difference between the two simulator types. However, comparing the influential variables, motivation, ease of use and concentration, the variable concentration contributes stronger to a difference between the two student groups than the other two variables. Simulator with time pressure Simulator without time pressure meanSDNmeanSDNFF crit Motivation3.890.74283.810.85270.1354.023 Ease of use4.141.67284.361.5270.3924.023 Concentration2.570.5282.650.49270.5144.023

15. INTED 2013 - Valencia (Spain) 4.-6. March 2013 Discussion  Regarding the first research question (differences concerning learning process), the necessary number of attempts to reach acceptable answers as well as the used time to finish the 16 scenarios were larger, although insignificantly larger, for the students who applied the simulator with time pressure. The time pressure did not lead to a shorter learning process. This confirms what has been found before [18], with the higher number of necessary attempts indicating that the time pressure may have contributed to more unacceptable answers which had to be revised.  The results for the second research question indicate virtually no relationship between the total learning time and the ease of use of the simulator and the students’ motivation

16. INTED 2013 - Valencia (Spain) 4.-6. March 2013 Discussion  There is an equally strong correlation with the perceived necessity to concentrate. The time pressure did obviously not cause a higher perception of necessary concentration. For the simulator with time pressure there is a negative correlation which means the higher the number of attempts, the lower the perceived necessity to concentrate, whereas there is a clear positive correlation for the simulator without time pressure which means the higher the number of attempts, the higher the perceived necessity to concentrate. Students make a connection between necessary concentration and the number of attempts, but they don’t perceive a connection between necessary concentration and the used time.  The negative correlation seems to reflect a “trial and error” mentality (leading to more attempts, but less perception of necessary concentration) for the students under time pressure. The goal to reflect reality as real as possible with the amount of time pressure chosen here seems to impede the learning process by encouraging students to guess.

17. INTED 2013 - Valencia (Spain) 4.-6. March 2013 Discussion  Although there was virtually no relationship between the total number of attempts and students’ motivation for students using the simulator without time pressure, students using the simulator with time pressure were more motivated when they needed more attempts. This seems to confirm the previous interpretation in that students motivation to score high led to a tendency of trying instead of increased concentration. This was not the case for the students using the simulator without time pressure. This confirms the earlier identified connection between heuristic decisions and time pressure [19].  Although the differences between the experimental group and the control group are insignificant regarding the influential variables, the results for the third research question (influence on learning process) indicate that the influence of the perceived necessity to concentrate is largest, followed by the ease of simulator use, and least regarding the students’ motivation.

18. INTED 2013 - Valencia (Spain) 4.-6. March 2013 Limitations  Construct validity may be limited, since analysis of the learning process is a difficult concept [19] and has been limited here to the learning time and attempts to reach an acceptable answer.  The ANOVA of pre-test replacing variables confirmed the internal validity. The experimental design, which included carrying out the experiment during normal class room times, avoided students dropping out except for the students who were missing on that day.  The maturation effect caused by familiarization was avoided by carrying out the experiment for all subjects of the same group at the same time.  Concerning the external validity it can be expected that the results of the study are to some degree representative for the category of students found here.  Since it was the first time for the students to apply this simulator, it may have caused a Hawthorne effect [20]

19. INTED 2013 - Valencia (Spain) 4.-6. March 2013 Conclusion  The results of this study showed no significant difference regarding the learning process of students who used the simulator with time pressure versus students who used the simulator without time pressure.  The findings seem to recommend caution when increasing time pressure in an attempt to reflect reality.  Increased time pressure resulted in more unacceptable answers which required students to revise their answers. It may increase students’ motivation to score high, but they seem to reach this by an increased mentality of “trial and error”.  This study can encourage engineering educators and institutions of engineering education to explore and implement ways to include time pressure in simulations since it has no significant detrimental effect on the learning process. However, caution is recommended regarding the amount of time pressure.

20. INTED 2013 - Valencia (Spain) 4.-6. March 2013 Conclusion  The results of this study showed no significant difference regarding the learning process of students who used the simulator with time pressure versus students who used the simulator without time pressure.  The findings seem to recommend caution when increasing time pressure in an attempt to reflect reality.  Increased time pressure resulted in more unacceptable answers which required students to revise their answers. It may increase students’ motivation to score high, but they seem to reach this by an increased mentality of “trial and error”.  This study can encourage engineering educators and institutions of engineering education to explore and implement ways to include time pressure in simulations since it has no significant detrimental effect on the learning process. However, caution is recommended regarding the amount of time pressure.