Situated agents must be able to interact with the physical environment that are located in with their conversational partner. Such an agent receives information both from its conversational partner and the physical world which it must integrate appropriately. Furthermore, since both the world and the language are changeable from one context to another it must be able to adapt to such changes or to learn from new information. Embodied and situated language processing is trying to solve challenges in natural language processing such as word sense disambiguation and interpretation of words in discourse as well as it gives us new insights about human cognition, knowledge, meaning and its representation. Research in vision relies on information represented in natural language, for example in the form ontologies, as this captures how humans partition and reason about the world. On the other hand, gestures and sign language are languages that are expressed and interpreted as visual information.
The masters thesis could be undertaken independently or as an extension of an existing project from the Embodied and Situated Language Processing (ESLP) course. Experience with dialogue systems and good Python programming skills is a plus.
Several projects are available subject of the approval of the potential supervisors. The main thread of the research would be how a linguistically inquisitive robot can update its representation of the world by engaging in dialogue conversation with a human. Sensory observations of a robot may be incomplete due to errors that robot's sensors or actuators introduce or simply because the robot has not explored and mapped the entire world yet. Can a robot query a human about the missing knowledge linguistically with clarification questions? Robotic view of the world is quite different from that of a human. How we can find a mapping between the representations that a robot builds using its sensors and the representations that are a result of human take on the world? The latter is challenging but necessary if robots and humans were to have a meaningful conversation.
Here are some suggested tasks:
A Lego robot, a miniature environment with blocks in a room
Microsoft Kinect or Microsoft robot studio, a table situation with objects
Generating route descriptions in a complex building
Grounded meaning representations
Earlier project (which this project could build on)
Simon Dobnik and other members of the Dialogue Technology Lab; for extracting ontological information also members of the Text Technology Lab