BIG: A Resource-Bounded Information Gathering Agent Victor Lesser, Bryan Horling, Frank Klassner, Anita Raja, Tom Wagner, Shelley Zhang Multi-Agent Systems Laboratory University of Massachusetts, Amherst

Multi-Agent Systems Lab, University of Massachusetts Talk Outline  Information Gathering problem.(Motivation)  The BIG Agent. l Interpretation. l Architecture & Components. l Sample Trace.  Performance Evaluation.  Integration Lessons & Future Work.

Multi-Agent Systems Lab, University of Massachusetts 3 Motivation  Rapid growth of WWW.  Growth has outstripped technology.  Information Retrieval technology a start. l Efficient, fast, general. l Access to enormous amount of data. (Alta Vista has indexed 125 million documents). l Browsing & processing documents manually non-trivial.

Multi-Agent Systems Lab, University of Massachusetts 4 The BIG Agent  BIG (resource Bounded Information Gathering) l Takes role of human in support of decision process. l Integration of Planning, scheduling, text processing and interpretation style reasoning. l Helps pick software packages.

Multi-Agent Systems Lab, University of Massachusetts 5 Sample Query  Input l Word processing package for a Mac. l $200 price limit. l Search process should take 10 min. & cost less than $5.

Multi-Agent Systems Lab, University of Massachusetts 6 The BIG Agent  Salient Features l Active search and discovery. l Resource Bounded Reasoning. l Goal-driven and Opportunistic control. l Information extraction and fusion..

Multi-Agent Systems Lab, University of Massachusetts 7 Sample Trace, Cont.  BIG recommends Corel WP3.5

Multi-Agent Systems Lab, University of Massachusetts 8 Information Gathering as Interpretation  Constructing high-level models from low-level data.  Information Gathering is an instance of this class. l Constructive problem solving. l Information fusion. l Sources of Uncertainty.  Tension between opportunism and planned action.

Multi-Agent Systems Lab, University of Massachusetts 9 BIG Agent Architecture

Multi-Agent Systems Lab, University of Massachusetts 10 BIG Components  Task Assessor l Forms initial plan, but not main planner. l Manages balance between opportunism & end-to-end.  Object & Server Database l Stores software product models l Models WWW sites. l Learns through persistence.  Document Classifiers l Distraction phenomenon caused by vendors.  Information Extractors l Builds/extracts structured data from unstructured text. l Extractors have varying tradeoffs and costs.

Multi-Agent Systems Lab, University of Massachusetts 11 TAEMS Task structure

Multi-Agent Systems Lab, University of Massachusetts 12 BIG Components, Cont.  TAEMS Modeling Framework l Domain-independent medium of exchange. l Hierarchical, statistically characterizes actions and alternatives.  Design-to-Criteria Scheduler Tradeoffs of different possible solution paths. l Builds custom schedules to meet a particular solution.  RESUN Planner l Blackboard interpretation planner. l Resolves sources of uncertainty. l Opportunistic problem solving.

Multi-Agent Systems Lab, University of Massachusetts 13 Sample Query  Input l Word processing package for a Mac. l $200 price limit. l Search process should take 10 min. & cost less than $5. l Product Quality attributes like usefulness, stability, ease of use, power features, etc.

Multi-Agent Systems Lab, University of Massachusetts 14 A Sample Trace Decision Maker Results & Supporting Data PlannerSchedulerExecutor Updates User RetrievesAssimilatesProcesses/Extracts Replans & Reschedules User Interface 132 Replans & Reschedules  Step 1: Task assessor forms skeletal plan.  Step 2: Plan scheduled by DTC scheduler.  Step 3: RESUN begins execution.

Multi-Agent Systems Lab, University of Massachusetts 15 Sample Trace, Cont.  Step 4: Queries issued l Parallel requests to MacZone (53) and Cyberian Outpost (61). l URLs returned used to build document-description info.  Step 5: 3 documents retrieved l Document length, recency, and site quality as criteria.  Step 6: Documents classified l Rejected children’s educational package for improving writing skills. l Rejected drawing/wp package by Corel. (dubious?)

Multi-Agent Systems Lab, University of Massachusetts 16 Sample Trace, Cont.  Step 7: 3 Text extractors execute l Produce Nisus Writer object. Product Name:Nisus Writer 5.1 Company Name:Nisus Price: $54.95 Processor:Mac Platform:Macintosh Processing Accuracy(Degree of Belief) range( ) GENRES = 0PRODUCTID=0.8COMPANYID=1.0 PRICE=1.0PROCESSOR=0.8DISKSPACE=0 PLATFORM=0.7

Multi-Agent Systems Lab, University of Massachusetts 17 Sample Trace, Cont.  Step 8-11: Gather more information. l Of remaining 111 document candidates, 4 are selected and retrieved, and classified. 2 are rejected. Extraction is highly uncertain & no new objects are produced.  Step 12-14: Processing new information. l 7 more document candidates are selected, retrieved, classified, and processed, producing 2 more objects:

Multi-Agent Systems Lab, University of Massachusetts 18 Sample Trace, Cont. Product Name:Corel WordPerfect 3.5 ACADEMIC Company name:Corel Price: $29.95 Platform: Mac/PwrMac Processing Accuracy(Degree of Belief): GENRES=0 PRODUCTID=0.8 COMPANYID=1.0 PRICE=1.0 PROCESSOR=0.8 DISKSPACE=0 PLATFORM=0.8 Product Name:Nisus Writer 5.1 Upgrade from 5.0 Company Name:Nisus Price:$29.95 Platform:Macintosh Processing Accuracy(Degree of Belief): GENRES=0 PRODUCTID=0.8 COMPANYID=1.0 PRICE=1.0 PROCESSOR=0.6 DISKSPACE=0 PLATFORM=0.8

Multi-Agent Systems Lab, University of Massachusetts 19 Sample Trace, Cont.  Step 15: Review gathering phase l Reviews retrieved, processed and extraction fills slots for Overall quality, usefulness, future usefulness. Ease of use, power features. Stability, enjoyability and value. l For each review, a pair is associated with the object.

Multi-Agent Systems Lab, University of Massachusetts 20 Sample Trace, Cont. Results & Supporting Data PlannerSchedulerExecutor Updates User RetrievesAssimilates Replans & Reschedules User Interface 12 4,57 Processes/Extracts Decision Maker  Step 16: Decision phase l Prune incomplete objects, discrepancy resolution. l Model includes number of products, coverage, quality, accuracy, and confidence of information.

Multi-Agent Systems Lab, University of Massachusetts 21 Sample Trace, Cont.  Step 17: BIG recommends Corel WP3.5

Multi-Agent Systems Lab, University of Massachusetts 22 Performance Evaluation

Multi-Agent Systems Lab, University of Massachusetts 23 Integration Lessons  Integration of the different AI problem solvers.  Backend processing for Information Extractor.  Integrated document classifier.  Modeling problems with the TAEMs.  Balance of goal driven and opportunisitic view.  Information fusion and Reasoning.  Learning component.

Multi-Agent Systems Lab, University of Massachusetts 24 Limitations and Future Work  Limitations: l Extraction is hard. l New domains require more training for extraction.  Future Work l More opportunism. l Decision confidence. l Multi-agent approach.

Multi-Agent Systems Lab, University of Massachusetts 25 Advantages of Document Classification

Multi-Agent Systems Lab, University of Massachusetts 26 Sample TAEMS task structure

Multi-Agent Systems Lab, University of Massachusetts 27 Related Work  “Moving Up the Food Chain”(Etzioni,AAAI 1996)  Meta Search Engines l Parallel queries, fast, coverage.  Personal Information Agents l Simple text processing, returns relevant list of URLs.  Shopping Agents l Specialized, price comparisons. World Wide Web Indices & Directories Agents & Softbots AltaVista, Yahoo Meta Crawler, Bargain Finder

Multi-Agent Systems Lab, University of Massachusetts 28 Strengths,Limitations and Future Work  Strengths: l Information extraction and fusion. l Incorporation of discovered information into process. l Representing and planning to resolve sources of uncertainty. l Ability to address deadlines and resource constraints. l Learning through experience.

Multi-Agent Systems Lab, University of Massachusetts 29 BIG Components, Cont.  Web Retrieval Interface l Gather URLs and interact with forms.  Document Classifiers l Distraction phenomenon caused by vendors.  Information Extractors l Builds/extracts structured data from unstructured text. l Extractors have varying tradeoffs and costs.  Decision Maker l Model of human decision process. l Considers preferences and confidence in information.