Universiteit van Amsterdam

Events

Institute for Logic, Language and Computation

Please note that this newsitem has been archived, and may contain outdated information or links.

1 February 2019, Cool Logic, Jack Harding

Speaker: Jack Harding
Title: Diagnostic Classifiers for Language Models
Date: Friday 1 February 2019
Time: 18:00-19:00
Location: ILLC seminar room F1.15, Science Park 107, Amsterdam

How do neural language models keep track of number agreement between subject and verb? We show that `diagnostic classifiers', trained to predict number from the internal states of a language model, provide a detailed understanding of how, when, and where this information is represented. Moreover, they give us insight into when and where number information is corrupted in cases where the language model ends up making agreement errors. To demonstrate the causal role played by the representations we find, we then use agreement information to influence the course of the LSTM during the processing of difficult sentences. Results from such an intervention reveal a large increase in the language model's accuracy. Together, these results show that diagnostic classifiers give us an unrivalled detailed look into the representation of linguistic information in neural models, and demonstrate that this knowledge can be used to improve their performance.As always, after the talk there will be beers and snack in the common room.

For more information, see https://events.illc.uva.nl/coollogic/talks/98 or contact Rachael Colley at .

Please note that this newsitem has been archived, and may contain outdated information or links.