Synthesizing Manipulation Sequences for Under-Specified Tasks using Unrolled Markov Random Fields

Many tasks in human environments require performing a sequence of navigation and manipulation steps involving objects. In unstructured human environments, the location and configuration of the objects involved often change in unpredictable ways. This requires a high-level planning strategy that is robust and flexible in an uncertain environment. We propose a novel dynamic planning strategy, which can be trained from a set of example sequences. High level tasks are expressed as a sequence of primitive actions or controllers (with appropriate parameters). Our score function, based on Markov Random Field (MRF), captures the relations between environment, controllers, and their arguments. By expressing the environment using sets of attributes, the approach generalizes well to unseen scenarios. We train the parameters of our MRF using a maximum margin learning method. We provide a detailed empirical validation of our overall framework demonstrating successful plan strategies for a variety of tasks.


Videos




Publications

Synthesizing Manipulation Sequences for Under-Specified Tasks using Unrolled Markov Random Fields, Jaeyong Sung, Bart Selman, Ashutosh Saxena. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2014. [PDF]

Learning Sequences of Controllers for Complex Manipulation Tasks, Jaeyong Sung, Bart Selman, Ashutosh Saxena. In ICML workshop on Prediction with Sequential Models, 2013. [PDF, arXiv]


People

Jaeyong Sungjysung at cs.cornell.edu
Prof. Bart Selmanselman at cs.cornell.edu
Prof. Ashutosh Saxenaasaxena at cs.cornell.edu