TTIC Distinguished Lecture Series - Christopher Manning

Title: Reading Comprehension and Natural Language Inference
Abstract: Much of computational linguistics and text understanding is either towards one end of the spectrum where there is no representation of compositional linguistic structure (bag-of-words models) or near the other extreme where very complex representations are employed (first order logic, AMR, HPSG, ...). A unifying theme of much of my recent work is to explore models with just a little bit of appropriate linguistic structure. I will focus here on recent case studies in reading comprehension and question answering, exploring the use of both natural logic and deep learning methods for reading comprehension and question answering.
Enabling a computer to understand a document so that it can answer comprehension questions is a central, yet still unsolved goal of NLP. I’ll first introduce our recent work on the Deepmind QA dataset - a recently released dataset of millions of examples constructed from news articles. On the one hand, we show that (simple) neural network models are surprisingly good at solving this task and achieving state-of-the-art accuracies; on the other hand, we did a careful hand-analysis of a small subset of the problems, and we argue that we are quite close to a performance ceiling on this dataset, and it is still quite far from genuine deep / complex understanding. I will then turn to the use of Natural Logic, a weak proof theory on surface linguistic forms which can nevertheless model many of the common-sense inferences that we wish to make over human language material. I will show how it can support common-sense reasoning and be part of a more linguistically based approach to open information extraction which outperforms previous systems. I show how to augment this approach with a shallow lexical classifier to handle situations where we cannot find any supporting premises. With this augmentation, the system gets very promising results on answering 4th grade science questions, improving over both the classifier in isolation, a strong IR baseline, and prior work. Finally, I will look at how we can incorporate more of the compositional structure of language, which is standardly used in logical approaches to understanding, into a deep learning model. I will emphasize some recent work which shows how that can be done quite efficiently by building the structure like a shift-reduce parser, and how the resulting system can produce stronger results than a sequence model on a natural language inference task.
The talk will include joint work with Gabor Angeli, Danqi Chen, and Sam Bowman.