BEGIN:VCALENDAR
VERSION:2.0
PRODID:ILLC Website
X-WR-TIMEZONE:Europe/Amsterdam
BEGIN:VTIMEZONE
TZID:Europe/Amsterdam
X-LIC-LOCATION:Europe/Amsterdam
BEGIN:DAYLIGHT
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
DTSTART:19700329T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:19701025T030000
RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
UID:/NewsandEvents/Archives/2024/newsitem/15210/16
 -October-2024-Computational-Linguistics-Seminar-Ve
 rna-Dankers
DTSTAMP:20241014T135730
SUMMARY:Computational Linguistics Seminar, Verna D
 ankers
ATTENDEE;ROLE=Speaker:Verna Dankers (University of
  Edinburgh)
DTSTART;TZID=Europe/Amsterdam:20241016T160000
LOCATION:Room L3.36, ILLC Lab42, Science Park 900,
  Amsterdam / Online
DESCRIPTION:Memorisation is a natural part of lear
 ning from real-world data: neural models pick up o
 n atypical input-output combinations and store tho
 se training examples in their parameter space. Tha
 t this happens is well-known, but which examples r
 equire memorisation and where in the millions (or 
 billions) of parameters memorisation occurs are qu
 estions that remain largely unanswered. In this ta
 lk, I first elaborate on the localisation question
  by examining memorisation in the context of class
 ification in fine-tuned PLMs, using 12 tasks. Our 
 findings give nuance to the generalisation-first m
 emorisation-second hypothesis dominant in the lite
 rature and find memorisation to be a gradual proce
 ss rather than a localised one. Secondly, I discus
 s memorisation from the viewpoint of the data usin
 g neural machine translation (NMT) models by putti
 ng individual data points on a memorisation-genera
 lisation map. I illustrate how the data points' ch
 aracteristics are predictive of memorisation in NM
 T and describe the influence that subsets of that 
 map have on NMT systems' performance.
X-ALT-DESC;FMTTYPE=text/html:\n  <p>Memorisation i
 s a natural part of learning from real-world data:
  neural models pick up on atypical input-output co
 mbinations and store those training examples in th
 eir parameter space. That this happens is well-kno
 wn, but which examples require memorisation and wh
 ere in the millions (or billions) of parameters me
 morisation occurs are questions that remain largel
 y unanswered. In this talk, I first elaborate on t
 he localisation question by examining memorisation
  in the context of classification in fine-tuned PL
 Ms, using 12 tasks. Our findings give nuance to th
 e generalisation-first memorisation-second hypothe
 sis dominant in the literature and find memorisati
 on to be a gradual process rather than a localised
  one. Secondly, I discuss memorisation from the vi
 ewpoint of the data using neural machine translati
 on (NMT) models by putting individual data points 
 on a memorisation-generalisation map. I illustrate
  how the data points' characteristics are predicti
 ve of memorisation in NMT and describe the influen
 ce that subsets of that map have on NMT systems' p
 erformance.</p>\n
URL:https://projects.illc.uva.nl/LaCo/CLS/
END:VEVENT
END:VCALENDAR