Programming by Demonstration with Situated Semantic Parsing

Abstract

Introduction Programming by Demonstration (PbD) is an approach to programming robots by demonstrating the desired behavior (Billard et al. 2008). Speech is a natural, hands-free way to augment demonstrations with control commands that guide the PbD process. However, existing speech interfaces for PbD systems rely on ad-hoc, predefined command sets that are rigid and require user training (Weiss et al. 2009; Akgun et al. 2012; Cakmak and Takayama 2014). Instead, we aim to develop flexible speech interfaces to accommodate user variations and ambiguous utterances. To that end, we propose to use a situated semantic parser that jointly reasons about the user’s speech and the robot’s state to resolve ambiguities. In this paper, we describe this approach and compare its utility to a rigid speech command interface.

BibTeX Entry

@inproceedings{artzi2014aaaifs,
  title = {Programming by Demonstration with Situated Semantic Parsing},
  author = {Artzi, Yoav and Forbes, Maxwell and Lee, Kenton and Cakmak, Maya},
  year = {2014},
  booktitle = {AAAI Fall Symposium Series: Artificial Intelligence for Human-Robot Interaction},
  type = {conference}
}