1. Why, What, and How to Log? Lessons from LISTEN Carnegie Mellon This work was supported in part by the National Science Foundation under ITR/IERI Grant No. REC-0326153 and in part by the Institute of Education Sciences, U.S. Department of Education, through Grants R305B070458, R305A080157, and R305A080628 to Carnegie Mellon University. The opinions expressed are those of the authors and do not necessarily represent the views of the Institute, the U.S. Department of Education, or the National Science Foundation. We thank the educators, students, and members of Project LISTEN who have helped generate, log, and analyze our data. I.Thou shalt not file; log to a database. II.Log user, computer, and start time. III.Include end time. IV.Index user, computer, and start time. V.Index thy parent to keep it handy. VI.Design databases to pool data from several servers. VII.Log each year’s data to its own database. VIII.Log each type of event to its own table. IX.Name fields consistently in and across databases. X.Log non-events carefully. TutoringReportingBrowsing Environment Raw input Interpreted input Student model Tutor decisions Events Tutor output # RowsTable Name 967storyTitleStory_LevelDirectoryAuthorCreation_DateStory_Text 11schoolNameAddressCityZipcodeStateCountryDistrict 39classSchool_NameClass_NameStatusClass_ID 545studentSchool_NameFirst_NameMiddle_NameLast_NameNicknameSexDate_Of_Birth 464,459student_clickMachine_NameStart_TimemsHelp_Start_TimeHelp_smsTypeTarget 9,792audio_recordingUser_IDMachine_NameStart_TimeEnd_TimesmsemsStory_Encounter_Start_Time 19,581word_swap_clickUser_IDMachine_NameStart_Timesms Word_Swap_Sentence _Encounter_Start_Time Word_Swap_Sentence _Encounter_sms Word 106,742multiple_choice_questionUser_IDMachine_NameStart_TimeEnd_TimesmsemsStory_Encounter_Start_Time 18,640text_editorUser_IDMachine_NameStart_TimeEnd_TimesmsemsStory_Encounter_Start_Time 9,379freeform_spoken_responseUser_IDMachine_NameStart_TimeEnd_TimesmsemsStory_Encounter_Start_Time 497,164utteranceMachine_Name Sentence_Encounter_Start_Ti me Sentence_Encounter _sms User_IDStart_TimeEnd_TimeWave_FilePath 1,769,011hypothesis_wordMachine_NameHypothesis_WordStart_TimeEnd_TimesmsemsUtterance_Start_Time 3,610,055aligned_wordMachine_NameUtterance_Start_TimeUtterance_smsUtterance_CountInteraction_TypeUser_ID Sentence_Encounter_Start_T ime 269,743student_word_dataSchool_NameCreation_DateUser_IDWordWord_KeyCount 8,115 student_word_production_ti me_grade_estimate School_NameUser_IDBinCount 78,173student_performanceSchool_NameUser_IDTimeMsMachine_NameWordStartTimeWordStartMs 79,832knowledge_tracingSchool_NameUser_ID Student_Performanc e_Time Student_Performanc e_Ms SkillPrior_LearnedPr_Learned 41,358event_responseMachine_NameUser_IDStart_TimesmsEnd_TimeemsIntervention_Directory 184,070help_givenMachine_NameDecided_TimedmsStart_TimesmsEnd_Timeems 3,273launchMachine_NameVersionLaunch_TimeLog_NameVolumesms 80crash_dumpMachine_NameTimeFile_NameDump 10,576sessionUser_IDStart_TimeEnd_TimeMachine_NameLog_NameExit_Throughsms 194,869scriptUser_IDScript_NameStart_TimeEnd_TimesmsemsMachine_Name 179,723set_variableMachine_NameTimemsUser_IDVariableValueScript_Start_Time 315,100stepUser_IDStep_NameStart_TimeEnd_TimesmsemsInterface_Object 21,817story_encounterStart_TimeEnd_TimeMachine_NameSession_Start_TimeExit_ThroughStory_DirectoryUser_ID 362,683sentence_encounterUser_IDMachine_NameStart_TimeEnd_TimesmsemsStory_Encounter_Start_Time 4,201word_builderUser_IDMachine_NameStart_TimeEnd_TimesmsemsStory_Encounter_Start_Time 26,589 word_builder_word_encount er User_IDMachine_NameStart_TimeEnd_TimesmsemsWord_Builder_Start_Time 11,728 word_swap_sentence_encou nter User_IDMachine_NameStart_TimeEnd_TimesmsemsStory_Encounter_Start_Time 1,985,366audio_outputMachine_NameStart_TimesmsAudio_TextUser_IDHelp_Start_TimeHelp_sms How to log: Why to log: What to log: Jack Mostow Carnegie Mellon University Joseph E. Beck Worcester Polytechnic Institute Project LISTEN www.cs.cmu.edu/~listen