OREGON STATE UNIVERSITY

You are here

A Bayesian Approach for Policy Learning from Trajectory Preference Queries

TitleA Bayesian Approach for Policy Learning from Trajectory Preference Queries
Publication TypeConference Paper
Year of Publication2012
AuthorsWilson, A., A. Fern, and P. Tadepalli
Conference NameAdvances in Neural Information Processing Systems (NIPS-2011)
Pagination1142–1150
Date Published12/2012
Conference LocationLake Tahoe, Nevada
Abstract

We consider the problem of learning control policies via trajectory preference queries to an expert. In particular, the learning agent can present an expert with short runs of a pair of policies originating from the same state and the expert then indicates the preferred trajectory. The agent's goal is to elicit a latent target policy from the expert with as few queries as possible. To tackle this problem we propose a novel Bayesian model of the querying process and introduce two methods that exploit this model to actively select expert queries. Experimental results on four benchmark problems indicate that our model can effectively learn policies from trajectory preference queries and that active query selection can be substantially more efficient than random selection.

URLhttp://books.nips.cc/papers/files/nips25/NIPS2012_0541.pdf