1. Work package 6: Integration into Educational Processes Work package leader UWS Professor Thomas M Connolly (UWS), Dr Liz Boyle (UWS) Dr Thomas Hainey (UWS), Dr Pablo Mareno Ger (UCM), Dr Jeffrey Earp (CNR), Dr Michela Ott (CNR), Dr. Dr Michael Kickmeier-Rust (TUG), Dr Francesco Bellotti (UniGe) 1

2. Aims and tasks of WP6 Aim: to examine how serious games (SGs) and serious virtual worlds (SVWs) can be integrated into formal educational systems T6.1 User and education stakeholder requirements (UWS) T6.2 Metrics for SG in education(UniGraz) T6.3 Integration methodologies(CNR-ITD) T6.4 Community of teachers and tutors (TU DELFT) 2

3. Year 1 activities: User and education stakeholder requirements Online longitudinal surveys: Scottish HE students (2005, 2007, 2009 and 2011) HE and FE students in Scotland HE students in the Netherlands Secondary school pupils in Greece Primary school teachers in Scotland Publications Hainey, T. H., Connolly, T. M., Stansfield, M., and Boyle, E. A. (2011). The Differences in Motivations of Online Game Players and Offline Game Players: A Combined Analysis of Three Studies at Higher Education Level. Computers & Education, 57, 2197-2211. Razak, A. A., Connolly, T. M. and Hainey, T. (2012). Teachers’ Views on the Approach of Digital Games-Based Learning within the Curriculum for Excellence, International Journal of Game-Based Learning, 2(1), 33-51. Hainey, T., Westera, W., Connolly, T. M., Boyle, E. A., Baxter, G., Beeby, R. B., & Soflano, M. (2013). Students’ attitudes toward playing games and using games in education: Comparing Scotland and the Netherlands. Computers & Education, 69, 474-484. 3

4. Year 2 activities: Integration methodologies Methodologies for integration of SGs in education context: guidelines, best practices, examples, assessment criteria, integration tools, evaluation. 1.Review of literature on integration of games in learning (including implementation experiences as well as empirical studies of outcomes) 2.Survey of GALA partners’ experiences: 2 approaches 4

5. Year 2 Activities: Integration methodologies Online survey: 6 experience reports from 4 partners –Discipline: 2 business, 1 cultural heritage, 1 history, 1 water management, 1 domain-independent –Kind of game: 2 “adventure“ games, 2 “management” games, 1 role-play, 1 simulation, 1 “collaborative scripting game” Structured interviews: 35 implementation experiences of Gala partners –Discipline: 11 business/commerce/management; 6 mathematics –Level: 19 university; 9 primary school; 6 adult education; 1 secondary school –Deployment settings: 11 classroom; 10 computer lab 5

6. Years 2 & 3: Collaboration activities Literature reviews - For year 2 the reviewers recommended that we change our approach in WP6.1 and WP6.2 from surveys to a meta-analysis: “A serious and formal meta-analysis associated with learning outcomes associated with serious games would be highly regarded by the global SG community” Books Psychology, Pedagogy and Assessment in Serious Games - 15 Gala partners Curriculum Integration of Serious Games - 9 Gala partners Case studies 6

7. Reason for the literature reviews Serious games for learning and behaviour change is a growth area There is an absence of STRONG empirical evidence about effectiveness of games Literature review can summarise the evidence and help to organise this fragmented research area Might confirm the value of games based learning? Interest in twin strands of engagement with games and learning with games

8. Years 2, 3 & 4: Literature reviews of empirical evidence about the learning outcomes of games 1. Systematic literature review 2004-2009: age 14+ (7,392 papers: 129: 70) Connolly, T. C., Boyle, E. A., Hainey, T. McArthur, E. & Boyle, J. M. (2012). A Systematic Literature Review of Empirical Evidence on Computer Games and Serious Games. Computers & Education, 59, 661 – 686. (over 200 citations) 2. Update to Systematic literature review - 2009-2014: age 14+ (54,580; 7,117 papers; 512: 144) Update of Connolly et al (2012) 3. Systematic literature review 2000-2013: age <14 (approx. 18,500 identified: 103 papers selected using the selection criteria: 45 papers quality coded over 9) 4. Quantitative meta-analysis of learning outcomes of games 8

9. Additional systematic reviews of games Systematic Literature Review of assessment in games Hainey, T., Connolly, T. M., Baxter, G. J., Boyle, L., Beeby, R. (2012). Assessment Integration in Games-based Learning. 6th European Conference on Games- based Learning (ECGBL), 4-5 October 2012, Cork, Ireland. Systematic Literature Review of engagement in games (66 citations) Boyle, E. A., Connolly, T. M., Hainey, T. & Boyle, J. M. (2012). Engagement in digital entertainment games: a systematic review. Computers in Human Behavior, 28, 771–780. A review of neurophysiological methods for monitoring brain activity in serious games and virtual environments Ninaus, M., Kober, S. E., Friedrich, E., Dunwell, I., de Freitas, S., Arnab, S., Ott, M., Kraycik, M., Lim, T., Louchart, S., Bellotti, F., Hannemann, A., Thin, A. G., Berta, R., Wood, G and Neuper (2012). Evaluating neurophysiological methods for monitoring brain activity in serious games and virtual environments. A review. 9

10. Year 3: Book, Psychology, Pedagogy and Assessment in Serious Games Psychological Aspects of Serious Games, Elizabeth A Boyle, Executive functions in Digital Games, Elizabeth A Boyle, James M E Boyle, Melody Terras, Judith Ramsey The road to critical thinking and intelligence analysis, Michel Rudnianski &Milos Kravcik Time factor assessment in Game Based Learning: Time Perspective and time-on- task as individual differences between players, Margarida Romero & Mireai Usart Neurofeedback and serious games, Manuel Ninaus & Lisa Freidrich Non-player characters and Artificial Intelligence, Rui Prada & Goncalo Pereira Games and social networks, Yulia Bachvarova Games For and By Teachers and Learners, Peter van Rosmalen, Amanda Wilson, Hans Hummel Assessment Integration in Games-Based Learning, Thomas Hainey & Thomas Connolly Evaluation of serious games: Approaches and challenges, Igor Mayer Evaluation of serious games: Case studies, Janickke Baulsruud Hauge et al 10

11. Year 3: Case Study of PLAYING HISTORY - THE PLAGUE CNR developed a common framework for structuring the selected cases analysed this case study with a view to definition of best practices and integration guidelines developed a set of supporting tools and technologies to help teachers use games in the classroom –teacher’s question matrix: aims of using game; audience for the game and characteristics of learners; features of games –game session monitoring sheet –adapted quiz used for pre and post tests 11

12. Year 3: ADDITIONAL CASE STUDIES Primary school La Dama Boba: aims to motivate young people to go to classical theatre plays, and to familiarize students with the plot and the characters before they go to the theatre CoLT2, Il Cambiamonete and Attenti al Cambio: cognitive rehabilitation games for using money; basic activities in the economic field Higher Education Lost in Space is used to teach students of Computer Science studies the XML mark-up language. CHERMUG – aimed to design, develop and test a game to support the teaching of introductory research methods and statistics to nursing and social science students –Staff and student support guides/ videos –Best practice case studies and use case scenarios 12

13. Projects arising from GALA collaboration 2012-2013: Continuing Higher Education In Research Methods Using Games (CHERMUG): a game for teaching Research Methods and Statistics; EU Lifelong Learning Programme, Development of Innovative ICT-based Content, Services, Pedagogies and Practices For Lifelong Learning (Key Activity 3); Transversal Programme: 2001KA3 ICT Multilateral Projects; Project reference: 519023-UK-2011-KA3MP. (€531,354) GALA partners: UWS, Complutense, PLAYGEN and OUNL 2015-2016: Youth Employability Training and Self-Development Framework (YOUTHYES); Erasmus + Strategic Partnerships addressing more than one field. KA2 – Cooperation and Innovation for Good Practices. 28652-LA-1-2014-1-UK-E4AKA1-ECHE. (€293,741). GALA partners: UWS and OUNL 13

14. Papers Boyle, E. A., Connolly, T. M. & Hainey, T. (2011). The role of psychology in understanding the impact of computer games. Entertainment Computing, 2, 69-74. Razak, A.A., Connolly, T.M. and Hainey, T. (2012). Teachers’ Views on the Approach of Digital Games-Based Learning within the Curriculum for Excellence, International Journal of Game-Based Learning, 2, 1, pp. 33-51 Hainey, T., Westera, W., Connolly, T. M., Boyle, L., Baxter, G., Beeby, R. B., and Soflano, M. (2013). Students’ attitudes toward playing games and using games in education: Comparing Scotland and the Netherlands. Computers and Education, 69. 474-484. Earp J., Ott M., Pozzi F. (2013), Facilitating educators' knowledge sharing with dedicated Information Systems, Computers in Human Behavior, 29 (2), 445-455, Elsevier, Paris (Paesi Bassi), Johnston B., Boyle, E., MacArthur, E., Fernandez Manion, B. (2013). The role of technology and digital gaming in nurse education. Nursing Standard. 27, 28, 35-38. Ney, M., Emin, V., Earp, J. Paving the Way to Game Based Learning: A Question Matrix for Teacher Reflection (2012). Procedia Computer Science, 15/2012. 17-24. ISSN 1877-0509, http://dx.doi.org/10.1016/j.procs.2012.10.053.http://dx.doi.org/10.1016/j.procs.2012.10.053 Baalsrud Hauge, J., Bellotti, F., Berta, R., Carvalho, M. B., De Gloria, A., Lavagnino, E., Nadolski, R., Ott, M. (2013). Field assessment of Serious Games for Entrepreneurship in Higher Education. Journal of Convergence Information Technology(JCIT), 8, 13. Ninaus, M., Kober, S. E., Friedrich, E., Dunwell, I., de Freitas, S., Arnab, S., Ott, M., Kraycik, M., Lim, T., Louchart, S., Bellotti, F., Hannemann, A., Thin, A. G., Berta, R., Wood, G and Neuper (2012). Evaluating neurophysiological methods for monitoring brain activity in serious games and virtual environments. A review. Boyle, J. M. & Boyle, E. A. (2014). Towards an understanding of the relationship between executive functions and learning outcomes from serious computer games, due to appear in Volume 8605 of the Lecture Notes in Computer Science series. 14

15. Conference Papers 1.Hainey, T., Connolly, T. M., Boyle, E. A., Azadegan, A., Wilson, A., Razak, A., & Gray, G. (2014). A Systematic Literature Review to Identify Empirical Evidence on the Use of Games-based Learning in Primary Education for Knowledge Acquisition and Content Understanding. Presentation at ECGBL 2014, Berlin. 2.Boyle, J. & Boyle, E. A. (2013). Towards an understanding of the relationship between executive functions and learning outcomes from serious computer games. Presentation at the GALA Conference, Paris, October 23-25 2013. 3.Hainey, T., Connolly, T. M., Baxter, G. J., Boyle, L., Beeby, R. (2012). Assessment Integration in Games-based Learning. 6th European Conference on Games-based Learning (ECGBL), 4-5 October 2012, Cork, Ireland. 4.Razak, A. A., Connolly, T. M., Baxter, G. J., Hainey, T., Wilson, A. (2012). The Use of Games-based Learning at Primary Education Level within the Curriculum for Excellence: A Combined Result of Two Regional Teacher Surveys. 6th European Conference on Games-based Learning (ECGBL), 4-5 October 2012, Cork, Ireland. 5.Wilson, A., Hainey, T., Connolly, T. M. (2012). Evaluation of Computer Games Developed by Primary School Children to Gauge Understanding of Programming Concepts. 6th European Conference on Games-based Learning (ECGBL), 4-5 October 2012, Cork, Ireland. 6.Scullion, J., Hainey, T., Stansfield, M. H., Connolly, T. M. (2012). A Pilot Implementation of an Immersive Online 3D Environment for Collaboration Among Computing Students in a Scottish University. 6th European Conference on Games- based Learning (ECGBL), 4-5 October 2012, Cork, Ireland. 15

16. Conference Papers 16 1.Razak, A.A., Connolly, T.M. and Hainey, T. (2011). “The Implementation of DGBL Approach in Primary Schools across Renfrewshire”, Games and Creativity in Education and Training (GACET’11), 17-18 November 2011, Rome, Italy. 2.Hainey, T., Connolly, T.M., Boyle, E.A. and Stansfield, M.H. (2011). “Motivations for Playing Computer Games in Relation to Gender: A Comparative Analysis of Three Studies at Higher Education”. 5th European Conference on Games-based Learning (ECGBL), 20-21 October 2011, Athens, Greece. 3.Razak, A.A., Connolly, T.M. and Hainey, T. (2011). “Games-Based Learning Application within the Curriculum For Excellence: From the Teacher’s Perspective”. 5th European Conference on Games-based Learning (ECGBL), 20-21 October 2011, Athens, Greece. 4.Hainey, T., Connolly, T.M., Boyle, E.A. and Stansfield, M.H. (2011). “Motivations for Playing Computer Games at Tertiary Education Level: A Comparison of Further Education and Higher Education Computing Students”. 5th European Conference on Games-based Learning (ECGBL), 20-21 October 2011, Athens, Greece. 5.Wilson, A., Connolly, T.M. and Hainey, T. (2011). “Evaluation of Introducing Programming to Younger School Children Using a Computer Game Construction Application”. 5th European Conference on Games-based Learning (ECGBL), 20-21 October 2011, Athens, Greece. 6.Scullion, J., Stansfield, M.H. and Connolly, T.M. (2011). “A Survey of Students’ Improved Mastery of Game Playing Skills Through Informal Online Game-Based Learning”, 5th European Conference on Games-based Learning (ECGBL), 20-21 October 2011, Athens, Greece.