To propose a methodology for constructing a wide coverage, state of the art NLI platform. To construct a small NLI platform buiding on this methodology that could be extended in the future.
Natural Language Inference (NLI), roughly put, is the task of determining whether an NL hypothesis can be inferred from an NL premise. Inferential ability according to Cooper et al. (1996) is the best way to test the semantic adequacy of NLP systems. In this context and given the importance of NLI to computational semantics, a number of NLI platforms have been proposed by the years, the most important ones being the FraCaS test suite, the Recognizing Textual Entailment (RTE) platforms and the Stanford NLI platform (SNLI). Despite their merits, all three of the platforms seem to concentrate on specific aspects of inference while NLI seems to be a much more complex phenomenon. The project will concentrate on tackling the needs of a wider coverage, both theoretically and implementationally, NLI platform.
- Learn about NLI and the three main platforms for it (FraCaS, RTE and NLI)
- Describe the merits as well as drawbacks of each platform from both a theoretical and practical perspective. Discuss any aspects of NLI that are not covered in these platforms
- Propose a methodology for constructing a state of the art NLI platform that will remedy the problems associated with earlier platforms. Justify the choices made.
- Construct a small NLI platform based on the proposed methodology that is machine readable. Discuss any potential challenges that platforms constructed using this methodology will cause to NLI systems.
- (optional) Implement an NLI system and evaluate against a part of your constructed test suite. Provide documentation for it.
- (optional) Evaluate current state of the art NLI systems against part or the whole constructed NLI platform. Discuss the results, ideas for improvement as well as the prospect of hybrid systems (combining both a machine learning/deep learning component as well as a symbolic (logical) component)
- Knowledge of semantics and pragmatics
- Programming skills, preferably Python in case of implemenation
- Knowledge of current techniques used in Machine Learning, Deep Learning and Logical approaches in case of evaluation
Supervisor: Stergios Chatzikyriakidis