Luxembourg January 28, 2013 European Commission Information Society and Media 2 nd Review Meeting GaLA Game and Learning Alliance The European Network of Excellence on Serious Games WP3 – SG Application Fields Michela Mortara, Chiara Eva Catalano, Bianca Falcidieno CNR – IMATI Marco Luccini CEDEP

Luxembourg January 28, 2013 European Commission Information Society and Media 2 nd Review Meeting Outline Overview of WP3 –mission and objectives Synthesis of Year 1 – activities and outcomes Year 2 –Focus, strategy, outcomes –Plan for user studies Plan for year 3 (4)

Luxembourg January 28, 2013 European Commission Information Society and Media 2 nd Review Meeting Overview of WP3 Mission: Understand the current practices, needs and desiderata of customers and developers of SGs in different application domains 6 specific Tasks, or Special Interest Groups –Business and Management (T3.1, CEDEP) –Engineering and Manufacturing (T3.2, POLIMI) –Health and Fitness (T3.3, HWU) –Security, Safety & Crisis Management (T3.4, NURC) –Humanities & Heritage (T3.5, CNR) –Personal & Social Learning & Ethics (T3.6, INESC-ID) 1 Harmonization & Integration Task(T3.7, CEDEP)

Luxembourg January 28, 2013 European Commission Information Society and Media 2 nd Review Meeting Synthesis of Year 1 Activity: surveying… SG proposition and diffusion dynamics Dissemination channels Key stakeholders Barriers to the adoption Requirements Open issues Outcomes: Collection of SGs in the application domains Best practices in adoption/development Recommendations for maximizing SG diffusion S S T T A A R R questionnaires, interviews

Luxembourg January 28, 2013 European Commission Information Society and Media 2 nd Review Meeting Main findings of year 1 High demand of evidence based practice Lack of a formal methodology to assess game effectiveness perceived as the main barrier to SG adoption Scepticism about SG effectiveness Low adoption, small market Uncertain return of investment Few skilled developers, low budget projects Scepticism about SG effectiveness Low adoption, small market Uncertain return of investment Few developers, low budget projects

Luxembourg January 28, 2013 European Commission Information Society and Media 2 nd Review Meeting Focus of Year 2 “The panel recommends that this WP develop a plan to design and conduct one or more empirical studies of SGs now in use which are generally regarded as effective but which lack empirical evidence of impact. Such a research plan could be developed in year two and conducted in year three” Based on the 1 st year outcomes and the reviewers’ recommendation, the focus of year 2 has been on the learning impact of SGs.

Luxembourg January 28, 2013 European Commission Information Society and Media 2 nd Review Meeting Strategy SG description AS – game testing and evaluation London – game workshops SG collection (year 1)SG monitoring (year 2) Game workshops Game testing and evaluation Collaborative SG descriptions Educational objectives & exploitation styles Technological features Availability to test Success –Diffusion / Popularity –Awards Learning Impact –Evidence from the literature –Recognised potential

Luxembourg January 28, 2013 European Commission Information Society and Media 2 nd Review Meeting Strategy SG description AS – game testing and evaluation London – game workshops SG collection (year 1)SG monitoring (year 2) Game workshops Game evaluation and testing Collaborative SG descriptions For each SIG: Best practices wrt maximizing the learning impact Current/best practices in the assessment methodology Identification of the SGs for user studies For each SIG: Best practices wrt maximizing the learning impact Current/best practices in the assessment methodology Identification of the SGs for user studies

Luxembourg January 28, 2013 European Commission Information Society and Media 2 nd Review Meeting BPs and recommendations on how to maximize and evaluate the Learning Impact [T3.7] Minimum cognitive overload Facilitate the learning Minimizing the cognitive overload R1 Engage the learner experientially/constructively Situate the learning Maximizing the learning experience Clear Evaluation strategies Comprehensive evaluation plan Designing learning evaluation as an Impact project R3 Flexibility, reusability, exploitation Facilitate the learning Design the SG as an integral part of a learning program R1 R2 R3 R4

Luxembourg January 28, 2013 European Commission Information Society and Media 2 nd Review Meeting BPs and recommendations on how to maximize and evaluate the Learning Impact [T3.7] Minimum cognitive overload Facilitate the learning Minimizing the cognitive overload R1 User friendly game interface & Intuitive navigation (Playing History, Plantville, Siqur) No previous knowledge required, gradually increasing difficult learning tasks (Shortfall, Thiatro) Use facilitation when the embedded complexity is relevant

Luxembourg January 28, 2013 European Commission Information Society and Media 2 nd Review Meeting BPs and recommendations on how to maximize and evaluate the Learning Impact [T3.7] R1 Situate the learning Engage the learner experientially/constructively Maximizing the learning experience R2 Minimization of cognitive overload a necessary condition A suitable environment and proper interactions to fit the context of use at best Immersive 3D environments (Thiatro, Boarders’ Ahoy!, TARGET, Innov8) Interaction among players (EagleRacing) Engage learners in building their new knowledge (Icura, MeTycoon) Fun but also emotional tension and frustration

Luxembourg January 28, 2013 European Commission Information Society and Media 2 nd Review Meeting BPs and recommendations on how to maximize and evaluate the Learning Impact [T3.7] Clear Evaluation strategies Comprehensive evaluation plan Designing learning evaluation as an Impact project R3 Facilitate the learning Design the SG as an integral part of a learning program R3 R4 Identify as early as possible the learning goals and the kinds of data to be collected in order to identify the suitable learning metrics. If the game is part of an educational strategy the stakeholders will be involved for the design and evaluation phase Facilitation to raise attention, provide learning guidance and consolidate the learning Maximization of the learning impact and effective evaluation

Luxembourg January 28, 2013 European Commission Information Society and Media 2 nd Review Meeting Plan for user studies – objectives Learning theories and models best practices & recommen dations Plan for user studies Selected SGs generally regarded as effective Go beyond available studies (if any) Validate the GALA evaluation (description template -WP1) Evaluate SG effectiveness per se / with respect to or coupled with other means Assess long term retention / actionability Not only cognitive gain, higher levels in Bloom’s / Kirkpatrick’s models

Luxembourg January 28, 2013 European Commission Information Society and Media 2 nd Review Meeting User studies EagleRacing (collaboration dynamics) MetaVals (finance) Seconds (supply chain management) Cosiga (product development process) SBCE (lean manufacturing) Re-Mission (support to treatment) Squire’s Quest (food and nutrition) Afghanistan pre-deployment course (defence, cultural awareness) Boarders Ahoy! (defence, ship boarding operations) Playing History – The Plague (History) ICURA (cultural awareness, japanese etiquette) Stop Disaster! (risk awareness) Darfur is Dying (human rights)

Luxembourg January 28, 2013 European Commission Information Society and Media 2 nd Review Meeting User studies EagleRacing (collaboration dynamics) MetaVals (finance) Seconds (supply chain management) Cosiga (product development process) SBCE (lean manufacturing) Re-Mission (support to treatment) Squire’s Quest (food and nutrition) Afghanistan pre-deployment course (defence, cultural awareness) Boarders Ahoy! (defence, ship boarding operations) Playing History – The Plague (History) ICURA (cultural awareness, japanese etiquette) Stop Disaster! (risk awareness) Darfur is Dying (human rights) Is part of some company specific programs and other education programs at CEDEP and at INSEAD The evaluation will benefit of a PGLC, never done before Is part of some company specific programs and other education programs at CEDEP and at INSEAD The evaluation will benefit of a PGLC, never done before

Luxembourg January 28, 2013 European Commission Information Society and Media 2 nd Review Meeting User studies EagleRacing (collaboration dynamics) MetaVals (finance) Seconds (supply chain management) Cosiga (product development process) SBCE (lean manufacturing) Re-Mission (support to treatment) Squire’s Quest (food and nutrition) Afghanistan pre-deployment course (defence, cultural awareness) Boarders Ahoy! (defence, ship boarding operations) Playing History – The Plague (History) ICURA (cultural awareness, japanese etiquette) Stop Disaster! (risk awareness) Darfur is Dying (human rights) Developed and adopted at POLIMI Summative evaluation including scenario questions Developed and adopted at POLIMI Summative evaluation including scenario questions

Luxembourg January 28, 2013 European Commission Information Society and Media 2 nd Review Meeting User studies EagleRacing (collaboration dynamics) MetaVals (finance) Seconds (supply chain management) Cosiga (product development process) SBCE (lean manufacturing) Re-Mission (support to treatment) Squire’s Quest (food and nutrition) Afghanistan pre-deployment course (defence, cultural awareness) Boarders Ahoy! (defence, ship boarding operations) Playing History – The Plague (History) ICURA (cultural awareness, japanese etiquette) Stop Disaster! (risk awareness) Darfur is Dying (human rights) Largely adopted and well studied BUT will be evaluated from the point of view of game mechanics and narrative Largely adopted and well studied BUT will be evaluated from the point of view of game mechanics and narrative

Luxembourg January 28, 2013 European Commission Information Society and Media 2 nd Review Meeting User studies EagleRacing (collaboration dynamics) MetaVals (finance) Seconds (supply chain management) Cosiga (product development process) SBCE (lean manufacturing) Re-Mission (support to treatment) Squire’s Quest (food and nutrition) Afghanistan pre-deployment course (defence, cultural awareness) Boarders Ahoy! (defence, ship boarding operations) Playing History – The Plague (History) ICURA (cultural awareness, japanese etiquette) Stop Disaster! (risk awareness) Darfur is Dying (human rights) Is part of education programs at CEDEP and at INSEAD and at some company specific programs. The evaluation will benefit of a PGLC, never done before Is part of education programs at CEDEP and at INSEAD and at some company specific programs. The evaluation will benefit of a PGLC, never done before Both considered BP for CA in defence operations – different technological solutions Planned a comparative study on different user profiles Both considered BP for CA in defence operations – different technological solutions Planned a comparative study on different user profiles

Luxembourg January 28, 2013 European Commission Information Society and Media 2 nd Review Meeting User studies EagleRacing (collaboration dynamics) MetaVals (finance) Seconds (supply chain management) Cosiga (product development process) SBCE (lean manufacturing) Re-Mission (support to treatment) Squire’s Quest (food and nutrition) Afghanistan pre-deployment course (defence, cultural awareness) Boarders Ahoy! (defence, ship boarding operations) Playing History – The Plague (History) ICURA (cultural awareness, japanese etiquette) Stop Disaster! (risk awareness) Darfur is Dying (human rights) Largely adopted in Danish schools. Evaluation at DIS (Genova) in march, to be repeated over time Largely adopted in Danish schools. Evaluation at DIS (Genova) in march, to be repeated over time

Luxembourg January 28, 2013 European Commission Information Society and Media 2 nd Review Meeting User studies EagleRacing (collaboration dynamics) MetaVals (finance) Seconds (supply chain management) Cosiga (product development process) SBCE (lean manufacturing) Re-Mission (support to treatment) Squire’s Quest (food and nutrition) Afghanistan pre-deployment course (defence, cultural awareness) Boarders Ahoy! (defence, ship boarding operations) Playing History – The Plague (History) ICURA (cultural awareness, japanese etiquette) Stop Disaster! (risk awareness) Darfur is Dying (human rights) Large diffusion (supported by U.N.) positive perception of learning impact but no empirical studies yet Evaluation of experimental vs control group Large diffusion (supported by U.N.) positive perception of learning impact but no empirical studies yet Evaluation of experimental vs control group

Luxembourg January 28, 2013 European Commission Information Society and Media 2 nd Review Meeting Further SIG activities (WP3 Handbook) Set-up and maintain dedicated SIG web pages (as suggested by the reviewers) [joint effort with WP8] Contribute to SG description Publish scientific papers [16 ref. proc, 2 ref. journals], organize Special Issues [1] and Workshops [3] Attract Associate Partners Monitoring SG proposition, stakeholders, dissemination channels

Luxembourg January 28, 2013 European Commission Information Society and Media 2 nd Review Meeting Next steps Main efforts in year 3: –Conduct the user studies –Disseminate the outcomes –Recommendations for maximizing the total value of SGs: revision, update, validation through the user studies –Continue Evaluating/describing SGs, Monitoring SG proposition, Updating the SIG pages Outlook to year 4: –describing and prioritizing the kinds of research that is needed to measure the learning impact (reviewers’ rec.) from the application perspective –SIG Final Thematic Workshops

Luxembourg January 28, 2013 European Commission Information Society and Media 2 nd Review Meeting Thank you for your attention