NLP Seminar Series

Join us online or on site at the AI Sweden Stockholm office at Fleminggatan 41. We welcome all attendees to work from the AI Sweden Stockholm office for the entire day.

Wednesday October 12, 15.00-16.30
Title: Reading between the lines: a better benchmark for text understanding
Speaker: Yoav Goldberg, Professor at Bar Ilan University and Research Director at AI2 Israel

Abstract: What does it mean to "understand" a text? Many recent NLP works take a question-answering (QA) perspective to this question, under which understanding a text means the ability to accurately answer questions about it. While this approach is useful, it can also be misleading: some questions are easier than others, some questions can be answered without access to the text at all, and, furthermore, questions often leak information about the answer. Thus, determining understanding based on question-answering ability alone is not sufficient. Indeed, many QA datasets are now considered "solved" by deep learning systems, while it should be clear that these systems do not really understand the text.

In this talk I highlight deficiencies of QA for measuring text understanding, and also introduce a new NLP task we devised---together with a large corresponding dataset---which I argue serves as a much stronger indicator of text understanding than QA is. Beyond being a benchmark for text understanding, the task is also linguistically grounded, and is a very useful building block for downstream text processing applications.

Yoav Goldberg will give his talk on site at the AI Sweden office on Fleminggatan 41, Stockholm.

Sign me up! 

By practitioners, for practitioners

Each seminar features an initial 45-minute presentation by practitioners or researchers followed by discussion. The NLP Seminar Series is organized by AI Sweden and RISE NLP Group.

Moderators:
Magnus Sahlgren, Head of Research, Natural Language Understanding at AI Sweden
Joakim Nivre, Head of Natural Language Processing at RISE.

Proposals for presentations can be sent to magnus.sahlgren@ai.se or francisca.hoyer@ai.se