David L Chen and
Raymond J Mooney.
2011.
Learning to Interpret Natural Language Navigation Instructions from Observations.. In
AAAI, vol
2, pp
1--2. cit
47. [
semparse, d=nav]
pdf url url abstract google scholar
The ability to understand natural-language instructions is crit- ical to building intelligent agents that interact with humans. We present a system that learns to transform natural-language navigation instructions into executable formal plans. Given no prior linguistic knowledge, the system learns by simply observing how humans follow navigation instructions. The system is evaluated in three complex virtual indoor environ- ments with numerous objects and landmarks. A previously collected realistic corpus of complex English navigation in- structions for these environments is used for training and test- ing data. By using a learned lexicon to refine inferred plans and a supervised learner to induce a semantic parser, the sys- tem is able to automatically learn to correctly interpret a rea- sonable fraction of the complex instructions in this corpus.
Matt MacMahon,
Brian Stankiewicz and
Benjamin Kuipers.
2006.
Walk the talk: connecting language, knowledge, and action in route instructions. In
proceedings of the 21st national conference on Artificial intelligence - Volume 2, pp
1475--1482,
Boston, Massachusetts.
AAAI Press. cit
78. [
semparse, d=nav]
url pdf abstract google scholar
Following verbal route instructions requires knowledge of language, space, action and perception. We present MARCO, an agent that follows free-form, natural language route in- structions by representing and executing a sequence of com- pound action specifications that model which actions to take under which conditions. MARCO infers implicit actions from knowledge of both linguistic conditional phrases and from spatial action and local configurations. Thus, MARCO per- forms explicit actions, implicit actions necessary to achieve the stated conditions, and exploratory actions to learn about the world.
We gathered a corpus of 786 route instructions from six peo- ple in three large-scale virtual indoor environments. Thirty- six other people followed these instructions and rated them for quality. These human participants finished at the intended destination on 69% of the trials. MARCO followed the same instructions in the same environments, with a success rate of 61%. We measured the efficacy of action inference with MARCO variants lacking action inference: executing only ex- plicit actions, MARCO succeeded on just 28% of the trials. For this task, inferring implicit actions is essential to follow poor instructions, but is also crucial for many highly-rated route instructions.