||In my talk I will present my Master thesis which I wrote at the University of Southern California about dialogue systems. So, let me introduce you to Eve, a dialogue agent which can play a fast-paced image matching game. The agent works together in a team with a human game partner to match as many images correctly as possible in a given time. Eve learned different interaction policies based on the work presented in my thesis. Her best policy allows her to play the game as efficient as human-human teams, leading to an average score which is not significantly different from those of the human-human teams.
The agentís system architecture is designed to operate in different degrees of incrementality to support the evaluation of incremental processing in dialogue interactions. Different machine learning techniques and input features have been evaluated to optimize Eveís language understanding.
My thesis also introduces a learning and offline evaluation framework which enables the agent to learn different dialogue strategies from a collected corpus of game interactions. In addition to the offline evaluation, different dialogue policies were evaluated in a large web study on Amazon Mechanical Turk with more than a hundred participants. Collecting data on the web is a very cheap and time-saving alternative to the lab. Therefore I will also introduce the Pair-Me-Up Framework with which we can collect interactions entirely on the web.