Environmental Adaptation Proposal for September 2006 CMU-Boeing Experiments

Motivation Engineering speech interfaces for robots operating in particular environments is difficult, and tends to result in brittle systems. We argue that linguistic adaptation is a required feature for cost-effective use of robots in most applications that desire primarily speech-based communication. Without it, interfaces will need to be either –Usable across a large set of domains (a Herculean task) –Or specifically designed for each application (cost effective for only the most common applications).

Environmental Adaptation Goals We desire interfaces that can adapt to operation in particular environments. Along with environmental adaptation (learning about specific environments), these systems must properly generalize and differentiate environments, facilitating the ability to context-switch between environments.

Environmental Examples Environments can be interpreted at many levels. For example: –Mobile indoor robotics. –After-hours indoor patrol. –Big-box store patrol. –Macy’s patrol. –Macy’s furniture-level patrol. Each of these environmental contexts may require some contextual sensitivity as well as generalizations. It is certainly conceivable that robots will need to switch from one context to the next in the course of a single application.

Methods Robots will track factors that define environments (eg location and objects) and learn in the context of those environments. Learning may happen at many levels, but we will focus on object expectations, which is an important aspect of both speech recognition and machine vision, and has the potential to greatly improve efficiency and accuracy.

Experiments We hope to show that use of the robots will be successively more efficient with each use, and that such efficiency gains are the result of robot learning, and do not depend on human adaptation to the situation. Successful demonstrations of environmental adaptation and use will require that experiments in which successive people’s use of the system in a common environment and with similar goals result in robot adaptation.

Groundwork The groundwork for such research requires a system that can –detect and refer to objects of multiple distinguishable kinds –infer an environment generalization from object discovery –and alter behavior for that environmental context. We propose a three-phase demonstration schedule starting this September corresponding to the points above.

Phase I: Detect and Refer Multiple fiduciaries will be introduced, of which the robots will be able to differentiate. Although the robots will be able to differentiate the objects, the names of the objects may be unknown to the system a priori. The robots will ask for object names, or otherwise infer them from the dialog.

Phase II: Environmental Inference The robots will associate references to objects with their locations upon reference. A clustering algorithm can be used to identify environmental borders.

Phase III: Context-appropriate Behavior Robots will exhibit behavior that is appropriate in the environmental context. –Language understanding priors (very important in speech recognition applications) and language generation will be appropriate for the environmental context. –We also hope to demonstrate some non- linguistic appropriate behaviors (the Segway will follow a bit more closely in the tightly- space “office environment”

Impact Successful linguistic environmental adaptation in a robot-neutral interface, as could be demonstrated in by experiment, has several benefits. Conceivably, an interface could be built for very general use, say indoor exploration, and then used effectively in diverse sub-domain applications, say bomb disposal or indoor patrol. The groundwork demonstrations proposed in these slides could position us to lead the development of a powerful tool.