1. Centralizing POMDP Solvers via Web Services BY Roi Ceren Muthukumaran Chandrasekaran

2. Background POMDP Models a single agent’s decision process in which the agent doesn’t directly observe the underlying state Exact solution yields the optimal action for each possible belief over the states Sequence of optimal actions is known as the optimal policy of the agent and can be represented as a decision tree In practice, POMDPs are often computationally intractable to solve exactly Thus, the need for approximate solutions Several exact and approximate methods for solving POMDPs exist No centralized test-bed available

3. Background Web services Applications can be broadcast as a service on a web server, such as Apache A wrapper, such as Axis or Tomcat, can be used to execute these applications Using standardized protocols, such as SOAP or REST, servers can communicate with one another to access these services In this way, we can allow disparate platforms to interoperate

4. Proposed Problem Many programs exist to implement POMDP resolutions, their subsets, and approximations techniques Robust applications streamlining and centralizing these functions and programs are largely not available We propose a web application that allows users to present problem domains and, in one location, use POMDP and various approximations to resolve them

5. Approach Expose at least one exact and one approximate POMDP programs on a THINC Lab PC Cassandra organization source code (exact) http://www.cassandra.org/pomdp/code/index.shtml http://www.cassandra.org/pomdp/code/index.shtml SARSOP via APPL toolkit (approximate POMDP planning) http://bigbird.comp.nus.edu.sg/pmwiki/farm/motion/index.php?n=Site.PomdpPlanning http://bigbird.comp.nus.edu.sg/pmwiki/farm/motion/index.php?n=Site.PomdpPlanning Publish access to these functions via web services SOAP or REST for communication Build a web interface that utilizes these web services

6. Approach Users will interact with the web interface to resolve POMDP problems 1.User provides a.pomdp or.pomdpx file via the user’s browser 2.File is passed to the POMDP solver with the selected parameters 3.Policy is retrieved from POMDP solver 4.Policy is displayed graphically as a policy tree in a graph applet via the user’s browser along with time of computation BrowserCommunication Protocol POMDP Solver.pomdp(x)/parameters policy

7. Approach (Web Interface) Input: AJAX Form POMDP file uploader Parameter input, examples Time limit Horizon Discount factor Stopping criteria Solver type Output: Applet Resulting policy tree Computation time POMDP File Location Upload POMDP S OLVER RESULTSRESULTS Browser time in ms Select Algorithm V A LGORITHM P ARAMETER 1 P ARAMETER 2 P ARAMETER 3 … Download Print

8. Approach (REST/SOAP) Apache Axis C++ module will be configured on the POMDP solver server Will serve as a wrapper for the various implementations of POMDP solvers Either REST or SOAP will be used to communicate REST calls will be made in JavaScript methods on the client side after form submission SOAP calls will be made using the PHP module kSOAP Policy data will be collected, parsed, and, using JavaScript, passed to the Applet

9. Demo Overview Website will be loaded during presentation Problem domain file will be uploaded and parameters configured.pomdp/x file Discount factor, horizon, etc. Applet will display output information Policy will be parsed as a decision tree Computation time will be overlaid

10. Demo Details Implement a working demonstration of the following methods Exact POMDPs Enumeration Incremental Pruning (if time permits) Approximate POMDPs SARSOP Sample Domains Tiger Problem

11. Deliverables Central location for toolkits Located on Muthu’s THINC lab machine e.g. Cassandra program/APPL toolkit Web interface for POMDP solver Form and decision tree applet URL is TBD on Muthu’s cs.uga.edu website

12. Responsibilities Muthu Configuring server with POMDP toolkits Web page design Roi Implementing RESTful web services Joint Web interface AJAX web forms Graph applet