BEGIN:VCALENDAR VERSION:2.0 PRODID:ILLC Website X-WR-TIMEZONE:Europe/Amsterdam BEGIN:VTIMEZONE TZID:Europe/Amsterdam X-LIC-LOCATION:Europe/Amsterdam BEGIN:DAYLIGHT TZOFFSETFROM:+0100 TZOFFSETTO:+0200 TZNAME:CEST DTSTART:19700329T020000 RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU END:DAYLIGHT BEGIN:STANDARD TZOFFSETFROM:+0200 TZOFFSETTO:+0100 TZNAME:CET DTSTART:19701025T030000 RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU END:STANDARD END:VTIMEZONE BEGIN:VEVENT UID:/NewsandEvents/Archives/2019/newsitem/10539/1- February-2019-Cool-Logic-Jack-Harding DTSTAMP:20190128T150101 SUMMARY:Cool Logic, Jack Harding ATTENDEE;ROLE=Speaker:Jack Harding DTSTART;TZID=Europe/Amsterdam:20190201T180000 DTEND;TZID=Europe/Amsterdam:20190201T190000 LOCATION:ILLC seminar room F1.15, Science Park 107 , Amsterdam DESCRIPTION:How do neural language models keep tra ck of number agreement between subject and verb? W e show that `diagnostic classifiers', trained to p redict number from the internal states of a langua ge model, provide a detailed understanding of how, when, and where this information is represented. Moreover, they give us insight into when and where number information is corrupted in cases where th e language model ends up making agreement errors. To demonstrate the causal role played by the repre sentations we find, we then use agreement informat ion to influence the course of the LSTM during the processing of difficult sentences. Results from s uch an intervention reveal a large increase in the language model's accuracy. Together, these result s show that diagnostic classifiers give us an unri valled detailed look into the representation of li nguistic information in neural models, and demonst rate that this knowledge can be used to improve th eir performance.As always, after the talk there wi ll be beers and snack in the common room. X-ALT-DESC;FMTTYPE=text/html:\n
How do neural language models keep track of number agreement bet ween subject and verb? We show that `diagnostic cl assifiers', trained to predict number from the int ernal states of a language model, provide a detail ed understanding of how, when, and where this info rmation is represented. Moreover, they give us ins ight into when and where number information is cor rupted in cases where the language model ends up m aking agreement errors. To demonstrate the causal role played by the representations we find, we the n use agreement information to influence the cours e of the LSTM during the processing of difficult s entences. Results from such an intervention reveal a large increase in the language model's accuracy . Together, these results show that diagnostic cla ssifiers give us an unrivalled detailed look into the representation of linguistic information in ne ural models, and demonstrate that this knowledge c an be used to improve their performance.As always, after the talk there will be beers and snack in t he common room.
URL:https://events.illc.uva.nl/coollogic/talks/98 CONTACT:Rachael Colley at rachaelhcolley at gmail. com END:VEVENT END:VCALENDAR