1. Tutoring and Educational Applications 2pm: Question Generation Based on Numerical Entities in Basque (Itziar Aldabe, Montse Maritxalar, Ander Soraluze) 2:25pm: A Graph Theory Approach for Generating Multiple Choice Exams (Sarah Luger) 2:50pm: Generating Mathematical Word Problems (Sandra Williams) 3:05pm: Moderated discussion (Jack Mostow) 1 AAAI Symposium on Question Generation, Nov. 4-6, Arlington, VA

2. Questions about Questions Target: what does it take to answer the question? Use: why ask the question? Question type: cloze? wh-/how/so/…? find/compare/…? Answer type: multiple choice? fill-in? open-ended? Generation: how to construct question, answer, distracters? Modality: menu? click? keyboard? speech? graphics? … Assessment: how to score answer? how to generate feedback? Evaluation: how well does question achieve use? how to tell? Discussion: – Why bother – why not just use cloze? – What kinds of difficulty are good, for what? – What’s novel? – What key idea(s) transfer? 211/4/2011

3. Question Generation Based on Numerical Entities in Basque Target: numerical fact (measure, date, time, number) Use: tests Question types: Which? When? How many? Answer type (planned): multiple choice Generation: find numerical entity; transform sentence Modality: unspecified Assessment: = correct answer? Evaluation (human): grammatical? fluent? Discussion: better than cloze? how? 311/4/2011

4. A Graph Theory Approach for Generating Multiple Choice Exams (Sarah Luger) Target: unspecified Use: tests Question type: unspecified Answer type: multiple choice Generation: none; given question, answer, and 4-5 distracters Modality: unspecified Assessment: correct choice? Evaluation: difficulty = fool more good than bad students Ideas: extract complete “virtual exams” from partial test data Discussion: What features make distracters difficult? What kinds of difficulty are good, for what uses? What can wrong answers reveal? Can Q-matrix or other learning identify types of students and questions? Relation to work on equating scores based on different item substs? 411/4/2011

5. Generating Mathematical Word Problems (Sandra Williams) Target: solve word problem Use: practice numeracy in realistic contexts Question type: multi-sentence problem involving two entities Answer type: number Generation: refactor and aggregate OWL, realize in English Modality: unspecified Assessment: = correct answer? Evaluation: none yet Ideas: Manipulate 5 difficulty factors: readability, irrelevant numbers, extraneous information, order of numbers, conceptual difficulty of math Discussion: What kinds of difficulty are good, for what uses? 511/4/2011