Programming by Demonstration with Situated Semantic Parsing

Y. Artzi, M. Forbes, K. Lee, and M. Cakmak, “Programming by Demonstration with Situated Semantic Parsing,” 2014.


Introduction Programming by Demonstration (PbD) is an approach to programming robots by demonstrating the desired behavior (Billard et al. 2008). Speech is a natural, hands-free way to augment demonstrations with control commands that guide the PbD process. However, existing speech interfaces for PbD systems rely on ad-hoc, predefined command sets that are rigid and require user training (Weiss et al. 2009; Akgun et al. 2012; Cakmak and Takayama 2014). Instead, we aim to develop flexible speech interfaces to accommodate user variations and ambiguous utterances. To that end, we propose to use a situated semantic parser that jointly reasons about the user’s speech and the robot’s state to resolve ambiguities. In this paper, we describe this approach and compare its utility to a rigid speech command interface.

BibTeX Entry

  title = {Programming by Demonstration with Situated Semantic Parsing},
  author = {Artzi, Yoav and Forbes, Maxwell and Lee, Kenton and Cakmak, Maya},
  booktitle = {2014 AAAI Fall Symposium Series: Artificial Intelligence for Human-Robot Interaction},
  year = {2014}