Integration of Search and Learning Algorithms Eugene Fink

Basic intuition An intelligent system should select or invent an effective approach to a given problem.

Challenges Select among available search algorithms (or invent a new algorithm) Restate the problem, to improve the efficiency of the selected algorithm

Search algorithm search problem

Search algorithm search problemsolution ?

Search algorithm search problemsolution ? A search module inputs a problem and runs until finding a solution, failing, or reaching a time limit.

Speed-up learning algorithm speed-up learning problem

Speed-up learning algorithm speed-up learning problem problem and new knowledge

Speed-up learning algorithm speed-up learning problem problem and new knowledge search

Speed-up learning algorithm speed-up learning problem An algorithm inputs a problem and generates additional knowledge, which improves search efficiency. problem and new knowledge search

… or static-analysis algorithm static analysis problem An algorithm inputs a problem and generates additional knowledge, which improves search efficiency. problem and new knowledge search

Goals Construct an architecture for integration of multiple search and learning modules Centralized architecture Integration tools

Goals Construct an architecture for integration of multiple search and learning modules Develop a control mechanism that chooses appropriate modules for each problem Centralized architecture Integration tools Top-level control decisions Reuse of accumulated knowledge

Related work AI systems with multiple learning and search modules: Soar, Prodigy, … Integration tools: Multiagent Planning Architecture (Wilkins and Myers), Plan++ (Yang et al.) Selection among available algorithms: Horvitz, Breese, Russell, Minton,...

Shaper system Three main parts: Library of search modules Library of learning modules Top-level control mechanism

System outline

Control center

System outline Manual control Automatic control Control center

System outline Manual control Automatic control Control center Given a new problem: Select appropriate modules Apply them to solve the problem Repeat if necessary

Features of the top-level control Fast: Less than a second per problem General: Independent of specific modules Allows participation of a human expert: Initial knowledge and interactive advice

Features of the automatic control Several learning mechanisms Symbolic and statistical techniques Combining exploitation and exploration

Automatic control Solve the problem or learn additional knowledge?

Automatic control Which learner to apply? Which past results to use? Solve the problem or learn additional knowledge?

Automatic control Which learner to apply? Which past results to use? Invoke the selected learning module Solve the problem or learn additional knowledge?

Automatic control Which learner to apply? Which past results to use? Solve or skip the problem? With which search module? Which learned data to use? Invoke the selected learning module Solve the problem or learn additional knowledge?

Automatic control Which learner to apply? Which past results to use? Solve or skip the problem? With which search module? Which learned data to use? Invoke the selected learning module Invoke the selected search module Wait for the next problem Solve the problem or learn additional knowledge?

Automatic control Which learner to apply? Which past results to use? Solve or skip the problem? With which search module? Which learned data to use? Invoke the selected learning module Invoke the selected search module Wait for the next problem Solve the problem or learn additional knowledge?

Performance example Solving a series of 50 problems, and learning which modules are the best. order of solving problems

Performance example Solving a series of 50 problems, and learning which modules are the best. order of solving problems

Performance example Solving a series of 500 problems, and learning which modules are the best. order of solving problems

Applications Past: AI planning (Prodigy) Present: Hardware design (power estimates) Future: Biomedical data, vision,...