1. Learning, Recognizing, and Assisting with Activities Tom Dietterich Oregon State University

2. Assumptions Goal: Integrated, autonomous, and useful AI systems –Must collaborate well with people Must recognize and understand human goals, attentional state, costs of coordination, etc.

3. Use Case 1: Edit and return document Document1 Person1 Attach Person2 Save Attachment Doc1.doc Message1 Send SaveAs Doc2.doc ReplyTo Attach Send AGENT could: automatically create TODO item when email arrives remind user when deadline is near detect when user has finished editing Doc2.doc and offer to send it back to Person1 automatically remove TODO item when completed

4. Use Case 2: NSF Proposal Review print /www.fastlane.nsf. gov/jsp/homepage /prop_review.jsp Request to review, includes proposal ID and password web page paste ID, password web page navigate download service/nsf/proposal.pdf /www.fastlane.nsf. gov/jsp/homepage /prop_review.jsp web page navigate web page navigate paste ID, password web page navigate web page submit fill out review form fastlane.nsf.gov web page logout open URL AGENT could: automatically create TODO item when email arrives remind user when deadline is near automatically login and download & print proposal automatically login and navigate to “review form” page automatically remove TODO item when completed confirmation page

5. Use Case 3: Prepare Quarterly Report Attach From: beers@sri.com g/darpa/calo/manage ment/arpa/Q3- report.doc SaveAs Send Save Attachment: Q3-report-chin.doc Save Attachment: Q3-report-williams.doc Save Attachment: Q3-report-sanchez.doc Paste Chin Williams Sanchez Save Attachment Attach ReplyTo g/darpa/calo/manage ment/arpa/Q3-report- template.doc Edit using WORD Send reminder AGENT could offer to automatically create TODO item when email arrives automatically save attachment and open it in Word automatically create outgoing email msg, address it to the correct recipients, and attach the template automatically track the email replies and save the attachments (with the right names) in the right folder automatically offer to send reminders to the missing subcontractors automatically open up the template and all replies in Word automatically attach the final file to a reply email to Melissa automatically delete TODO item when complete

6. Research Challenges Representing Workflows Learning Workflows Recognizing Workflows Deciding (Learning) When and How to Help

7. Representing Workflows For what purpose: –execution: sequence of actions (possibly with conditionals and iteration) –recognition: partially-ordered sequence of actions (with conditionals and iteration) capture additional features to aid recognition (e.g., email speech acts) –learning: need action models to detect unobserved steps and understand goals –assistance need action models to understand goals Workflow steps: –informational inputs (file name, file, URL) –action (click Download) –action models (creates file on disk with file name; contents = contents of URL file)

8. Representing Workflows commentOnDocument :- mailArrived(EmailRID, Requester, SpeechAct, Deadline, [Attachment1]), outlookOpen(EmailRID), attachmentSave(EmailRID, Attachment1, FileRID), wordEditDocument(FileRID, EditedFileRID), outlookOpen(EmailRID), outlookComposeReply(NewEmailRID, EmailRID), outlookSendReply(NewEmailRID, Requester, SpeechAct2, [Attachment2]), outlookAttachmentInfo(NewEmailRID, EditedFileRID, Attachment2). wordEditDocument(FileRID, FinalRID) :- wordOpen(FileRID), finishEdit(FileRID, FinalRID). // simply close the file and return it finishEdit(FileRID, FileRID) :- wordClose(FileRID). // close the file, then later re-open it and continue finishEdit(FileRID, FinalRID) :- wordClose(FileRID), wordOpen(FileRID), finishEdit(FileRID, FinalRID). // perform a SaveAs and then continue finishEdit(FileRID, FinalRID) :- wordSaveAs(FileRID, NewFileRID), finishEdit(NewFileRID, FinalRID).

9. Learning Workflows Learning by Demonstration –LAPDOG: PBD system at SRI –Lau, et al. at IBM and before that UW –PLOW: Allen et al. Rochester Learning by Observation (unsupervised) –Weld et al.

10. Recognizing Workflows Challenges on the desktop –Multiple workflows interleaved –Multiple instances of the same workflow interleaved reviewing multiple NSF proposals –Sharing across workflows log in and navigate only once, then download multiple files –Unmodeled background events IM, nytimes.com, weather.com, etc.

11. Recognition Task Given: –a set of workflow schemas –an observation sequence Find: –all instances of those workflow schemas in the observation sequence –detect each instance as early as possible –report the current state of all active workflow schemas at each point in time Metrics: –false positives, false negatives, timeliness

12. Assistance What steps can the AGENT do? What steps should the AGENT do? How and when should the AGENT coordinate with the user? Decision-theoretic collaboration –model the user’s intentions and attentional state –estimate the expected benefit of AGENT’s assistive plan (including coordination cost) –choose action that maximizes expected benefit

13. Rich Intention Structures Goal stack –traditional programming languages –hierarchical reinforcement learning formalisms –cognitive architectures: SOAR, ACT-R Goal graph –ABL (Mateas) The user’s TODO list is an intention structure –so is the Inbox for many people Revised statement of our goal: –representation, learning, recognition, and assistance with rich intention structures

14. Related Topics Argumentation and Persuasion –How do two agents exchange information in order to reach agreement? Explanation-based Teaching and Learning –AGENT makes a mistake –user says “Why did you do that?” –AGENT explains –user corrects parts of the explanation –etc. Transfer Learning –How do I transfer to you something I’ve learned when you have a different ontology I can’t give you all of my training data (privacy, bandwidth)?

15. Summary Goal: AI AGENT that can help humans Prerequisite: AGENT must understand what its user is doing