1. Plan Recognition in Virtual Laboratories Ofra Amir and Yaakov (Kobi) Gal Ben-Gurion University of The Negev Department of Information Systems Engineering

2. Background Educational software in the sciences is including open-ended construction environments Students build and analyze models using these software 2

3. Virtual Labs – An interactive simulation of a chemistry lab. 3

4. Virtual Labs – Dilution Problem Your supervisor has just asked you to prepare 500mL of 3M HNO 3 for tomorrow's experiment. In the stockroom explorer, you will find a cabinet called Stock Solutions. Open this cabinet to find a 2.5L bottle labeled 15.4M HNO 3. 4

5. The Problem Please prepare a flask containing 500 ml of a 3M solution… Im confused Boring… This is easy! During class: 5

6. The Problem After class: I wonder how each student performed… 6

7. The Goal Develop automatic support for teachers in their analysis of student performance – Approach: use plan recognition to infer students problem solving plans 7

8. Related Work Bayesian networks to model students interactions with intelligent tutors [Conati et al., 2002] Complete algorithms for plan recognition (CSP) [Reddy et al, 2009; Quilici et al, 1998] Heuristic algorithm for recognizing students activities in pedagogical software for statistics [Gal et al, 2008] 8

9. Solution Building blocks User actions Recipes Plans 9

10. User Actions – Log Files 10

11. Recipes [Pollack, 1990] Basic actions are rudimentary (log actions) Complex actions are abstract A recipe for a complex action describes – A series of sub-actions for completing the action – Constraints on these actions 11

12. Representing actions: Recipe structure: The Recipe Language Action Name An action parameter with value constraint D[dt:H20, sid_1,did_1][ 0 ] Parameter without value constraint Action Pre-conditions D[sid_1,did_2] -> D[sid_1,did_1][ ] C[sid_2,did_2][ ] The complex action sub-actions did_1=sid_2 Constraints on sub- actions parameters 12

13. Plans in Virtual Labs A plan for a complex action is – A hierarchy of recipes towards completing the action The plan represents students activity with the software Plan recognition – Infer students activities based on their actions with the software, given a set of recipes 13

14. Plan Recognition Approach Plan Recognition Algorithm Student Plan 14

15. Dilution Problem Recipes To solve the problem, the student should: – Pour H 2 0 to the destination flask – Pour HNO 3 to the destination flask Students can solve this problem in many different ways 15

16. Dilution Problem - Example Solution 1 2 1 16

17. Dilution Problem - Example Solution 2 17

18. Challenges in The Virtual Labs Domain Indefinite repetition of activities Interleaving activities Trial-and-error, mistakes Conclusion: – Complete approach intractable for this domain 18

19. Build Plan Greedy Algorithm Input: – A set of Virtual Labs basic actions, A set of recipes Algorithm steps: – initialize open list with the actions from the log – for each recipe in order of increasing depth find match (Recipe, open list) while match exists – add complex action to the open list – create branches from the complex action to its sub-actions – remove sub-actions from open list – Call find match with the updated open list 19

20. ( SM, did=ID5, sid=ID1 ) ( MO, id=ID5 ) ( SM, did=ID6, sid=ID5 ) ( MO, id=ID1 ) ( AS, id=ID6 ) ( MO, id=ID6 ) ( MO, id=ID5 ) ( FC, did=ID6,sid=ID5 ) ( SM, did=ID6, sid=ID5 ) ( C, did=ID5, sid=ID1 ) ( MO, id=ID5 ) ( AS, id=ID6 ) ( MO, id=ID6 ) ( MO, id=ID5 ) ( FC, did=ID6,sid=ID5 ) ( C, did=ID6, sid=ID5 ) ( C, did=ID5, sid=ID1 ) ( MO, id=ID5 ) ( AS, id=ID6 ) ( MO, id=ID6 ) ( MO, id=ID5 ) ( FC, did=ID6,sid=ID5 ) ( C, did=ID6, sid=ID5 ) ( D, did=ID6, sid=ID1 ) P1P1 P2P2 P3P3 20 Recipe: C -> SM SMRecipe: D -> C C

21. Find Match Find match searches for actions in the open list which fulfill the recipe for the complex action Actions in the match can be free ordered as long as they satisfy the constraints in the recipe Was implemented as depth first search, but can be implemented in other ways Find Match is complete, given a recipe and an open list of actions 21

22. Dilution Problem – Partial Plan 22 MSC – Mixing solution component MSI – Mixing solution through intermediate flask SDP – Solve Dilution Problem

23. Empirical Evaluation The algorithm was run on 20 log files taken from real student interactions – 6 different problems – logs ranged in size from 20 actions to 187 actions – Plans ranged in depth from 3 to 14 levels The plans were validated by a domain expert 23

24. Contributions A new computationally efficient plan recognition algorithm that can cope with interleaving activities, mistakes, indefinite repetition. The algorithm can be integrated with real pedagogical software Shown to succeed on real-world data 24

25. Questions 25